rust,mmap , How to create and write to memory mapped files?


How to create and write to memory mapped files?

Question:

Tag: rust,mmap

I'm trying to create a memory mapped file using std::os::MemoryMap. The current approach looks as follows:

use std::os;
use std::ptr;
use std::old_io as io;
use std::os::unix::prelude::AsRawFd;
use std::os::MapOption;

let path = Path::new("test.mmap");

let f = match io::File::open_mode(&path, io::Open, io::ReadWrite) {
    Ok(f) => f,
    Err(err) => panic!("Could not open file: {}", err),
};

let mmap_opts = &[
    MapOption::MapReadable,
    MapOption::MapWritable,
    MapOption::MapFd(f.as_raw_fd())
];

let mmap = match os::MemoryMap::new(1024*1024, mmap_opts) {
    Ok(mmap) => {
        println!("Successfully created the mmap: {}", mmap.len());
        mmap
    }
    Err(err) => panic!("Could not read the mmap: {}", err),
};

unsafe {
   let data = mmap.data();

    if data.is_null() {
        panic!("Could not access data from memory mapped file")
    }

    let src = "Hello!";
    ptr::copy_memory(data, src.as_ptr(), src.as_bytes().len());
}

This program fails with

Process didn't exit successfully: `target/mmap` (status=4)

when calling ptr::copy_memory or any other operations on data.


Answer:

You are missing two main parts:

  1. mmap doesn't allocate any space on its own, so you need to set some space in the file. Without this, I get Illegal instruction: 4 when running on OS X.

  2. MemoryMap is private by default so you need to mark the mapping as public so that changes are written back to the file (I'm assuming you want the writes to be saved). Without this, the code runs, but the file is never changed.

Here's a version that works for me:

#![feature(path,io,os,std_misc,core,libc)]

use std::os;
use std::ptr;
use std::old_io as io;
use std::os::unix::prelude::AsRawFd;
use std::os::MapOption;

extern crate libc;

fn main() {
    let size = 1024*1024;

    let path = Path::new("test.mmap");
    let mut f = io::File::open_mode(&path, io::Open, io::ReadWrite).unwrap();

    // Allocate space in the file first
    f.seek(size as i64, io::SeekStyle::SeekSet).unwrap();
    f.write_u8(0).unwrap();
    f.seek(0, io::SeekStyle::SeekSet).unwrap();

    let mmap_opts = &[
        // Then make the mapping *public* so it is written back to the file
        MapOption::MapNonStandardFlags(libc::consts::os::posix88::MAP_SHARED),
        MapOption::MapReadable,
        MapOption::MapWritable,
        MapOption::MapFd(f.as_raw_fd()),
    ];

    let mmap = os::MemoryMap::new(size, mmap_opts).unwrap();

    let data = mmap.data();

    if data.is_null() {
        panic!("Could not access data from memory mapped file")
    }

    let src = "Hello!";
    let src_data = src.as_bytes();

    unsafe {
        ptr::copy_memory(data, src_data.as_ptr(), src_data.len());
    }
}

Related:


Casting to a generic type


rust
I have a newbie question about generics in Rust (version 1.0). Let's say I write a generic function to do division. Never mind the usefulness of such a function; it's a simple function to keep this question simple. fn divide<T: std::ops::Div>(a: T, b: T) -> T { a / b...

Result has no method called “unwrap()”?


rust
What a strange error: let res1 = get_res(); assert!(res1.is_ok()); assert_eq!("just for test", res1.unwrap()); //error The error is: type `core::result::Result<(MyStruct1, collections::btree::map::BTreeMap<collections::string::String, collections::string::String>), Error>` does not implement any method in scope named `unwrap` ...

Rust: Lifetime of String from file [duplicate]


file,io,rust
This question already has an answer here: Return local String as slice 1 answer I'm trying to read in some external GLSL code into Rust. The reading works properly, but I run into a lifetime issue in the final expression (in the Ok(_) branch) error: s does not live...

Why does Drop take &mut self instead of self?


rust
Why does Drop’s method have signature fn drop(&mut self) instead of fn drop(self)? This makes it difficult to move values out of the fields e.g. self.join_handle.join() or std::mem::drop(self.file) (error: cannot move out of type X, which defines the Drop trait).

Modify Collection While Iterating Over it


rust
Gist to source Disclaimer: I'm just starting to learn Rust and I know this is not the best way to do this. I'm simply playing around to see what I can and cannot do. I'm also trying to limit any copying to restrict myself a little bit. I have a...

Reading immutable value inside spawned thread


rust
I'm trying to make this example work use std::sync::mpsc::{Sender, Receiver}; use std::sync::mpsc; use std::thread; use std::sync::Arc; struct User { reference: String, email: String } struct UserIndex { reference: usize, email: usize } fn main() { let rows = vec![ vec!["abcd", "[email protected]"], vec!["efgh", "[email protected]"], vec!["wfee", "[email protected]"], vec!["rrgr", "[email protected]"] ]; let user_index...

Load a shared library linked to Rust library in R


r,shared-libraries,rust
Following up on this question here, I am having issues using dyn.load to load a shared library that is linked to a Rust dylib. I suspect it has something to do with where R is looking for the Rust dylib, but I have not found a way to specify another...

Conditionally compile only one module at a time


rust
Sometimes while refactoring a change in one of the modules has impact on every other module (or many others), because say a type has been changed or a function parameter has been changed etc. In this case instead of changing everything and then compiling at once is there a way...

Indexing a String


rust
I want to perform a very simple task, but I cannot manage to stop the compiler from complaining. fn transform(s: String) -> String { let bytes = s.as_bytes(); format!("{}/{}", bytes[0..2], bytes[2..4]) } [u8] does not have a constant size known at compile-time. Some tips making this operation to work as...

Asm x86 segmentation fault in reading from file


assembly,x86,segmentation-fault,mmap
I'm receiving segmentation fault in 5th line of loop2. This is the part of my code which is reading values from mmapped file byte by byte. Could you please tell me what I'm doing wrong? .global _start _start: mov $2,%rax mov 16(%rsp),%rdi mov $02,%rsi syscall cmp $0,%rax jl exit 1:...

Pass Python list to Rust function


python,rust,ffi
I have a Rust library that needs to be imported into Python via the ctypes module. My goal is to use Rust functions that take Vec<T> / i32 as arguments and return those types, from Python. Currently, I can pass integers to the Rust functions, and have them return lists...

“conflicting implementations for trait” when trying to be generic


types,rust
Background: I'm using the nalgebra library and I want to create a structure that represents a multivariate normal distribution. M is the type of the matrix, e.g. Mat4<f64>. My current attempt looks like this: use std::ops::Mul; use std::marker::PhantomData; use nalgebra::*; #[allow(non_snake_case)] pub struct Multivar𝒩<N, V, M: SquareMat<N, V>> { μ:...

Filter vector in place


iterator,rust
In Rust is it possible to operate on a vector in-place? I'd like to remove some elements from a Vec, but vec.iter().filter().collect() creates a new vector with borrowed items. I'd like to mutate the original Vec without extra memory allocation (and keep memory of removed elements as an extra capacity...

How to use multiple variables in routes with Nickel?


rust,nickel
Nickel states that you can use variables in the URLs, which sounds very useful, but is it possible to use multiple variables? Something like: www.example.com/login/:userid?:apikey?:etc server.get("/start/:userid?:passwd", middleware! { |request| // format!("This is user: {:?} = {:?}", // request.param("userid"), // request.param("passwd") // ); }); ...

Slow memory allocation in OSX


c,osx,mmap
I'm trying to trace down a memory allocation problem I have in OSX. If I compile and run the following code normally, it will run pretty fast. #include <sys/mman.h> #define SIZE 8 * 1024 * 1024 int main(int argc, char const *argv[]) { for (int i = 0; i <...

Recovering from `panic!` in another thread


rust
I know that in Rust there is no try/catch, and you can't throw a rolling save from the thread that is currently panicking. I know you should not create and handle errors like this. This is just for example's sake. However, I am wondering what the best way to recover...

Implementing a generic conversion from an object implementing the `Error` trait


rust
I cannot get the following code to compile. I get an error that From is already implemented. If I remove the manual impl of From I get the error that From is not implemented. If I do not implement Error it works fine. I suppose that this is due to...

How can I create Haskell-like functional dependencies


rust
Background: I'm using the nalgebra library and I want to create a structure that represents a multivariate normal distribution. The number and row type is uniquely determined by a square matrix type, so I want to write something like this: #[allow(non_snake_case)] pub struct Multivar𝒩<M: SquareMat<N, V>> { μ: V, Σ:...

Cannot infer appropriate lifetime for autoderef in Iterator impl


rust
I am trying to implement the Iterator trait for a struct which basically just acts as a borrower of an array of i32 values. However, I keep running into the compiler complaining about not being able to infer a lifetime inside the next method. I am aware of this answer,...

How can I send a function to another thread?


multithreading,unit-testing,rust
I am attempting to write a simpler unit test runner for my Rust project. I have created a TestFixture trait that my test fixture structs will implement, similar to inheriting from the unit test base class in other testing frameworks. The trait is fairly simple. This is my test fixture...

vector method push_all is not found for a custom struct


rust
So in this simple example #![feature(collections)] struct User { reference: String, email: String } fn main() { let rows = vec![ vec!["abcd".to_string(), "[email protected]".to_string()], vec!["efgh".to_string(), "[email protected]".to_string()], vec!["wfee".to_string(), "[email protected]".to_string()], vec!["rrgr".to_string(), "[email protected]".to_string()] ]; let mut rows_mut: Vec<Vec<String>> = Vec::new(); rows_mut.push_all(&rows); let mut users_mut: Vec<User> = Vec::new(); let users = vec![ User { reference:...

Immutable reference after mutable borrow


rust
I've been reading about rust for the past week and (trying) to play around with it. It seems I run into similar problems related to ownership/borrowing every time I use it, so here is the simplest piece of code that sort of illustrates the usual problems. use std::cell::RefCell; struct Res...

Stack behavior when returning a pointer to local variable


stack,rust
I have a simple example where the behaviour of Rust does not match my mental image, so I am wondering what am I missing: fn make_local_int_ptr() -> *const i32 { let a = 3; &a } fn main() { let my_ptr = make_local_int_ptr(); println!("{}", unsafe { *my_ptr } ); }...

Implement Debug trait for large array type


rust,traits
I gather that Rust provides Debug impl's for arrays size 32 and smaller. I also gather that I could implement Debug on a larger array by simply using write! with a very long format specifier. But I'm wondering if there's a better way. What is the recommended method for implementing...

Return a moving window of elements resulting from an iterator of Vec


iterator,rust
I'm trying to figure out how to return a window of elements from a vector that I've first filtered without copying it to a new vector. So this is the naive approach which works fine but I think will end up allocating a new vector from line 5 which I...

Global feature gates in Cargo


rust,rust-cargo
I would like to enable a feature gate for my entire Cargo project. For example, I would like #![feature(non_ascii_idents)] added to every source file. Is there a place to list them in Cargo.toml?

Calling an impl method from another impl method


rust
It seems I can't call the method of the same struct in Rust or maybe I don't understand something: struct St1 { aa: String } impl St1 { pub fn method1() -> String { //.... method2() //error: unresolved name method2 } pub fn method2() -> String { //.... } }...

Cargo dependency version syntax


rust,rust-cargo
Is there a page documenting the different Cargo syntax for dependencies? So far I have seen three... [dependencies] crate = "1.0.0" # I think this is an exact version match crate = "^1.0.0" # I think this means "use that latest 1.x.x" crate = "*" # I think this means...

Value does not live long enough when using multiple threads


rust
This is a simplified example of my situation: use std::sync::mpsc::{Sender, Receiver}; use std::sync::mpsc; use std::thread; struct User { reference: String, email: String } fn main() { let rows = vec![ vec!["abcd", "[email protected]"], vec!["efgh", "[email protected]"], vec!["wfee", "[email protected]"], vec!["rrgr", "[email protected]"] ]; let chunk_len = (rows.len() / 2) as usize; let mut chunks...

Struct vs enum lifetime differences


rust,lifetime
Why does this work #[derive(Debug)] pub struct Foo<'a,'b> { s : &'a str, n : &'b i32 } #[test] fn test_struct() { let f = Foo { s : &"bar" , n : &17 }; println!("{:?}",f); } but this doesn't #[derive(Debug)] pub enum Bar<'a,'b> { Baz ( &'a str), Fub...

Situations where Cell or RefCell is the best choice


rust
When would you be required to use Cell or RefCell? It seems like there are many other type choices that would be suitable in place of these, and the documentation warns that using RefCell is a bit of a "last resort". Is using these types a "code smell"? Can anyone...

More convenient way to work with strings in winapi calls


string,winapi,rust
I'm looking for more convenient way to work with std::String in winapi calls in Rust. Using rust v 0.12.0-nigtly with winapi 0.1.22 and user32-sys 0.1.1 Now I'm using something like this: use winapi; use user32; pub fn get_window_title(handle: i32) -> String { let mut v: Vec<u16> = Vec::new(); v.reserve(255); let...

the type of this value must be known in this context


rust
I want to get a length of a string which I've split: fn fn1(my_string: String) -> bool { let mut segments = my_string.split("."); if segments.collect().len() == 55 { //...... } // error: the type of this value must be known in this context How can I fix that error?...

How can I open a file with the standard text editor?


rust
My console based program has a config file which can be opened and edited in a text editor. How would I open said config file with the default text editor? I know there is std::process, but I can't get it to work. The program should be able to run on...

Can I create a macro that unrolls loops?


macros,rust
I'm trying to write some fast matrix code in Rust and to do this needs to ensure that loops are unrolled. Is there a way to create a compile-time for-loop? E.g: I want unroll_loop!(f, a, 3); to generate f(a, 0); f(a, 1); f(a, 2); ...

What is the idiomatic way to write a linked list with a tail pointer?


linked-list,rust,reference-counting
As a learning project for Rust, I have a very simple (working, if incomplete) implementation of a singly linked list. The declaration of the structs looks like this: type NodePtr<T> = Option<Box<Node<T>>>; struct Node<T> { data: T, next: NodePtr<T>, } pub struct LinkedList<T> { head: NodePtr<T>, } Implementing size and...

Create shared C object linked to Rust dylib for use in R


c++,c,r,rust,ffi
I am trying to create a shared object I can load into R that calls Rust functions by way of R's C API. To call Rust from C, I am following this blog post. My problem arises when I try to create the shared library and link to the Rust...

Create a vector from iterating hashmap


rust
What is the optional way in Rust to iterate a HashMap and collect the result into a Vec? This is my attempt so far: use std::collections::HashMap; struct User { reference: String, email: String } fn main() { let mut users: HashMap<String, User> = HashMap::new(); users.insert("first".to_string(), User { reference: "ref1".to_string(), email:...

Why do I need to use self::core::ops?


rust
I'm trying to use Mul from core. This is suggested by the compiler and works: extern crate core; use self::core::ops::Mul; but why doesn't extern crate core; use core::ops::Mul; work? I get the error error: unresolved import `core::ops::Mul`. Did you mean `self::core::ops`?...

Why do I have to expose a macro implementation's 'use' in the client library?


macros,rust
I'm trying to use a macro I've created in a separate module. With reference to this SO question, I've imported a macro fine. However it seems that I have Update to add macro implementation lib.rs #![macro_use] use std::fmt; use std::ffi::CString; use std::ffi::CStr; use std::str; extern crate libc; pub enum DbaxError...

How to implement variable increment via side effect?


rust,side-effects
For learning purposes, I tried this solution, but it does not work: use std::ops::Add; fn inc<T: Add>(x:&mut T) { *x += 1; } fn main() { let mut x:i32 = 10; let mut y:u8 = 1; inc(&mut x); inc(&mut y); println!("{} {}", x, y); } Error message: <anon>:4:5: 4:7 error:...

Bug in FFI when passing CString followed by an int


windows,rust,32-bit,ffi
My Rust test code extern "C" { fn test_int_only(n : libc::c_int); fn test_int_and_str(s : CString , n : libc::c_int); } pub fn test1() { unsafe { test_int_only(0); test_int_only(1); test_int_only(2); test_int_only(4); test_int_only(-12); } } pub fn test2() { unsafe { test_int_and_str(CString::new("Foo").unwrap(),0); test_int_and_str(CString::new("Bar").unwrap(),1); test_int_and_str(CString::new("Baz").unwrap(),2); test_int_and_str(CString::new("Fub").unwrap(),4); test_int_and_str(CString::new("Bub").unwrap(),-12); } } My C code void...

Renaming a crate with a hyphen doesn't compile anymore


rust
This no longer compiles: extern crate "my-crate" as my_crate1; What's the new syntax? I did not find anything by searching....

Can Rust handle cyclic data structures without any garbage collector?


rust
Is it possible to completely avoid a garbage collector and manual deallocation? Is it possible to implement an interpreter for a language that needs garbage collection (say, Scheme) in Rust, without implementing or using any garbage collector? ...

What is the most appropriate way to convert nibbles to a u64?


rust
I am trying to learn Rust and decided to write a program that converts a hex string into a u64. Currently, I have parsed the string into a vector of u8 values, each representing four bits (or "nibble"). I wrote the following code to take a Vec<u8> and return a...

Lifetime of a mutable element in struct


rust
How can I define a mutable element in a struct? If I have a following example struct User<'a> { reference: String, email: String, addresses: &'a mut Vec<Address> } struct Address { street: String, city: String } fn main() { let mut users = Vec::new(); users.push(User { reference: "ref".to_string(), email: "[email protected]".to_string(),...

Why is `Sized` bound necessary in this trait?


rust,traits
I have a simple trait with two associated functions like this: trait WithConstructor : Sized { fn new_with_param(param: usize) -> Self; fn new() -> Self { Self::new_with_param(0) } } The default implementation of the second method (new()) force me to put the Sized bound on the type. Why ? (I...

Writing Vec to a file


rust
I'm having trouble on writing Vec<u16> content to a file. #[derive(Debug, Copy, Clone, PartialEq)] pub enum ImageFormat { GrayScale, Rgb32, } #[derive(Debug, Copy, Clone, PartialEq)] pub struct ImageHeader { pub width: usize, pub height: usize, pub format: ImageFormat, } pub struct Image { pub header: ImageHeader, pub data: Vec<u16>, }...

remove duplicates from vector of custom struct


rust
I'm trying to remove duplicates in the below example: struct User { reference: String, email: String } fn main() { let mut users: Vec<User> = Vec::new(); users.push(User { reference: "abc".into(), email: "[email protected]".into() }); users.push(User { reference: "def".into(), email: "[email protected]".into() }); users.push(User { reference: "ghi".into(), email: "[email protected]".into() }); users.sort_by(|a, b| a.email.cmp(&b.email));...

cannot move out of borrowed content - ref doesn't work [duplicate]


rust
This question already has an answer here: “cannot move out of borrowed context” and “use of moved value” 1 answer I have this code: enum MyEnum1 { val1, val2 } struct Struct1 { field1: MyEnum1, field2: String } fn fn1(a: Struct1, b: String, c: String) { let let1 =...