memory,memory-management,operating-system,cpu-architecture , memory segments and physical RAM [closed]

memory segments and physical RAM [closed]


Tag: memory,memory-management,operating-system,cpu-architecture

The memory map of a process appears to be fragmented into segments (stack, heap, bss, data, and text),


In a modern OS supporting virtual memory, it is the address space of the process that is divided into these segments. And in general case that address space of the process is projected onto the physical RAM in a completely random fashion (with some fixed granularity, 4K typically). Address space pages located next to each other do not have to be projected into the neighboring physical pages of RAM. Physical pages of RAM do not have to maintain the same relative order as the process's address space pages. This all means that there is no such separation into segments in RAM and there can't possibly be.

In order to optimize memory access an OS might (and typically will) try to map sequential pages of the process address space to sequential pages in RAM, but that's just an optimization. In general case, the mapping is unpredictable. On top of that the RAM is shared by all processes in the system, with RAM pages belonging to different processes being arbitrarily interleaved in RAM, which eliminates any possibility of having such "segments" in RAM. There's no process-specific ordering or segmentation in RAM. RAM is just a cache for virtual memory mechanism.

Again, every process works with its own virtual address space. This is where these segments can exist. The process has no direct access to RAM. The process doesn't even need to know that RAM exists.


clEnqueueNDRangeKernel fills up entire memory

I am currently trying to write an OpenCL application doing some memory intensive calculations. To track the progress of all the calculations I created a for loop which creates different kernel groups. Unfortunately, the calculation fills up my whole memory. My guess is that the kernels are not done executing...

Is it safe to read and write on an array of 32 bit data byte by byte?

So I have a void * data of 32 bit unsigned integers which represents the pixels. Is it okay for me to access one of the pixels with a char * and modify the values directly? Or is it better to store my new pixel in a temporary uint32_t variable...

Fast memory allocation for real time data acquisition

I have a range of sensors connected to a PC that measure various physical parameters, like force, rotational speed and temperature. These sensors continuously produce samples at some sample rate. A sample consists of a timestamp and the measured dimension itself; the sample rates are in magnitudes of single-digit kilohertz...

Looking into dynamic variables created using pointers?

I was wondering if there is a tool in any IDE which can be used to "look" into dynamically allocated variables in C. If for example I use the code below, it is not possible to use the watches (CodeBlocks) to track the variables, only the adresse the pointer is...

Operator Overloading without memory copies

Imagine we have the type A and we want to do the following operation: A operator -(const A& a, const A& b) { A c; //... return c; } When we call this operator, we do something like this: A c; c = a - b; Basically we are allocating...

free() and mxFree() in MATLAB - freeing memory twice

Good day, I have the following code which has given me problems for a day already. I have debugged it, and it works fine until trying to free the memory. The free() function should be called at the end of the execution automatically, so I commented the mxFree() code out,...

Using a data pointer with CUDA (and integrated memory)

I am using a board with integrated gpu and cpu memory. I am also using an external matrix library (Blitz++). I would like to be able to grab the pointer to my data from the matrix object and pass it into a cuda kernel. After doing some digging, it sounds...

Simple explanation about loading files into memory,visual-studio-2010,memory-management
What does it mean to "load into memory"? I watched some videos online where they were talking about reducing processing reports times, and they constantly say "you don't want to load a huge file into memory, instead read line by line." Can I have an example of what loading a...

SetProcessWorkingSetSize does not work in compiling 64bit

I use the following command to reduce the memory usage of my program, I'm actually testing it, only when I compile the program in 64bit this command does not work fot, no error occurs, only the memory in task managerIt does not decrease too, since in compiling 32bit works perfectly,...

I am getting error in this code as “invalid indirection”

I am trying to dynamically allocate a contiguous block of memory, store some integer value and display it. #include<stdio.h> #include<conio.h> void main() { int i; int *ptr; ptr=(void *)malloc(sizeof(int)*5); //allocation of memory for(i=0;i<5;i++) { scanf("%d",&ptr[i]); } for(i=0;i<5;i++) { printf("%d",*ptr[i]); //error is found here`` } } } ...

Good approach to clean application data programmmatically in Android

When I use RecyclerView and/or Picasso. It seems to cache and store alot of data in my application. However, I want to delete the stored data and the cache in a simple way whenever I am exiting my app. What is the simplest way to delete all data stored in...

Scrapy Memory Error (too many requests) Python 2.7

I've been running a crawler in Scrapy to crawl a large site I'd rather not mention. I use the tutorial spider as a template, then I created a series of starting requests and let it crawl from there, using something like this: def start_requests(self): f = open('zipcodes.csv', 'r') lines =...

Explanation of why allocating a large array results in segmentation fault in C [duplicate]

This question already has an answer here: Segmentation fault on large array sizes 5 answers Memory allocation for global and local variables 3 answers I'm not too experienced in C, but I've been recently writing some program in the language to speed it up a little bit (it was...

Changing Django code logs me out of application

I am new to Django. I wrote a basic application. When I test it, every small change I make in the Python code logs me out from localhost. This happens when I use this cache backend: 'BACKEND': 'django.core.cache.backends.locmem.LocMemCache', but does not when I use this one: 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache', Is there...

Does the dart VM impose restrictions on the stack memory size of a native extension?

I'm learning to write a native extension and I noticed an odd occurrence. When I allocate a multidimensional array and access it like so: (excuse any messy C code and bad practices I might be using unless they're likely the cause of my error. My C is not-so-great) int table[rows][cols];...

QEvent ownership

I want to send both predefined and custom QEvent to a Qt application. Is the event required be allocated on the heap or can I just pass the pointer to a object on the stack? QKeyEvent stackevent(QEvent::KeyPress); QKeyEvent* heapevent = new QKeyEvent(QEvent::KeyPress); QEvent * event; event = &stackevent; //valid ??...

The range of virtual memory address in userspace

In linux, the range of virtual memory address in userspace, in other words, the range of value returned by malloc, is same as entire 64bits virtual memory space? Or, is there any sub-range of 64bits virtual memory space which is guaranteed not to be seen in userspace? Answers for UNIX...

Does Python copy references to objects when slicing a list?

When a list is sliced, are the references to its contents copied from the original list? I can imagine that this may not be necessary, but I read the opposite (mentioned in passing). This question matters for instance for the following idiom, in the case of a very long my_list:...

In Spark, what is left in the memory after a Job is done?

I used ./bin/spark-shell to run some experiments and find out the following facts. When running jobs (transformation + action), I notice the memory usage in the top. For example, for 5G text file, I did a simple filter() and count(). After the job is done, there are 7g marked as...

Assigning memory dynamically or on the stack based on a condition (if statement)

I have a C program in which an array of floats has its elements accessed quite often for the duration of the program. The size of the array depends on an argument that a user will input and therefore will vary. Generally, the size will be small enough (~ 125...

Converting collection to array with no extra memory

I need to convert Map into 2D array, so I've wrote this piece of code but it eats a lot of memory and I can't figure out why. private DataItem[][] convertDataToArrays(boolean[] filter, Map<Integer, List<T>> dataSet) double[] data = new double[sizeOfNewVector]; DataItem[][] reducedData = new DataItem[dataSet.size()][]; for (int i = dataSet.size()...

uninitialized data segment of program memory

In Uninitialized Data Segment, the kernel initialize the uninitialized variable to 0. But if we directly use an uninitialized variable (for eg. int sum; and not int sum=0; ) and use it in program like sum=sum+n(n be any number), sum will take any garbage value. So why the sum will...

How are Strings created and stored in Java?

To understand how String objects are created and stored, I tried the following program and I see output against which I have query. Can somebody please help? package corejava.immutable; public class StringTester { public static void main(String[] args) { // TODO Auto-generated method stub String s1 = "Omkar Patkar"; String...

Detemine memory used by Hazelcast cache

I'm implementing the backend of a game in Java and I was told to use Hazelcast to cache some data from the database. How could I find out how much memory is used by the cache (necessary to determine the business cost of this solution)?

Memory management in game development?

I'm new to android and i started to develop a game, I thought about memory management when loading images and using them for animations, I want to know how to manage it properly so it wont damage the game's smoothness, How could i proceed with this?. And while I'm asking...

Memory Issue for Array Conversion

If we convert a large array containing 0 and 1 as boolean to another array containing 0 and 1 as float, the size of array would be almost 10 times larger. What is the best way (if any) to handle this issue in python (Numpy) if we need this conversion?

How to get current memory page size in C#

I need to determine the memory page size in C# without using "Kernel32.dll" SYSTEM_INFO si; GetSystemInfo(&si); This is very important because the code should be cross platform and we should not have platform specific code. Is there any .NET class who provides that data? Thanks, Valentin...

Java - allocation in enhanced for loop

I have small problem in java with allocation of my object in "enhanced for loop". I tried something like this, which gives me later nullPointerException: SomeClass tab[] = new SomeClass[100]; for( SomeClass x : tab){ x = new SomeClass(); } So... Earlier I code in C++ and It was possible...

Program Runs Fine for Hours and Eventually Seg Faults with Memory Address 0x10 [closed]

So I've been tracking some memory issue for the longest time. I'm coding in C++ and I can see that my program mostly works. I am monitoring my resources and I dont think I have a memory leak because my memory used stays below 12% (I'm on a system with...

C++11 Allocation Requirement on Strings

I had heard that C++11 was going to require strings to be allocated in contiguous memory. I even thought I saw a stack overflow question on it, but I can't seem to find it. I know that in practice both gcc and Visual Studio do allocate strings contiguously, I'm just...

sort runs out of memory

I'm using a pipe including sort to merge multiple large textfiles and remove dupes. I don't have root permissions but the box isn't configured in any way to cut non root privileges further down than default debian jessie. The box has 32GB RAM and 16GB are in use. Regardless on...

memory content not erased after deleting my pointer (on a simple example) [duplicate]

This question already has an answer here: Can a local variable's memory be accessed outside its scope? 16 answers Here is a simple of example of a question I have: I create a pointer of integer (value 5), I print the pointer (hence the address) of the memory case...

Why I cannot use “fgets” to read a string to an element of my Struct?

I'm trying to create a program almost like "bank" using struct, but when the program should have read the string (variable "nome" that is name in portuguese) it totally ignore the "fgets" that I used. This is the part that I was talking about : printf("\nNome: \n"); fgets(vet[cont+1].nome, sizeof(vet[cont+1].nome), stdin);...

Getting many memory errors when try to run it for few days in my web crawler [on hold]

I am developing a web crawler application. When i run the program i am getting these error messages below: i've got these errors after running the program for more that 3 hours. I tried to allocate memory by changing eclipse.ini setting to 2048 MB of ram as it was answered...

Memory error in set resize

Here's a structure of set that I've written: struct state_set { struct state ***state_array; size_t *slot_sizes; size_t *slot_memory; }; Here's an initializer: struct state_set *state_set_init() { struct state_set *new_set = malloc(sizeof(struct state_set)); new_set->state_array = malloc(ARRAY_SIZE * sizeof(struct state**)); new_set->slot_sizes = malloc(ARRAY_SIZE * sizeof(size_t)); new_set->slot_memory = malloc(ARRAY_SIZE * sizeof(size_t)); for (int...

c++ dynamically declared array fails to work

I am trying to read the file's data into a dynamically declared array, using double *data = new double[14141414]() declaration. Note, it is rather a large file; hence large size of an array. The problem is I can not put all the data into an array as somewhere around index=14000000...

System.Management, Management Object Searcher, and RAM

I am brand new to coding and I'm having a go at trying to make a simple app that shows computer information in a multiline textbox, as you do. So, I managed to get the basics by searching the registry for os name, version, etc and using system.environment to get...

Heroku RAM not increasing with upgraded dynos

I have a massive function i have been calling manually through the heroku rails console. I have been receiving the error rapid fire in my logs: 2015-06-22T14:56:42.940517+00:00 heroku[run.9877]: Process running mem=575M(112.4%) 2015-06-22T14:56:42.940517+00:00 heroku[run.9877]: Error R14 (Memory quota exceeded) A 1X dyno is suppose to have 512 MB of RAM. I...

Is there a difference between new List() and new List(0)

Is there a difference between new List<T>() and new List<T>(0) ? May be this is a micro-optimization, but the idea is to understund the difference in term of memory allocation....

function to get 2d-arrays from stack and heap

Consider a simple struct, Foo, which contains a pointer to some other strucrt, Baz. and a two-dimensional array, (stack-allocated): Foo arr[ARR_SIZE][ARR_SIZE]; Next, consider a function which mallocs (in the heap, obviously) a memory for a new 2d-array of Foos. A code snippet: Foo** arr_copy = malloc(sizeof(Foo*) * ARR_SIZE); for (int...

Java how to limit number of threads acting on method

I have java method in my web application doing heavy file operation. The thing is, if more than 5 threads come simultaneously (which will come in testing phase) it breaks down. I mean it cannot handle heavy traffic. That's why I want to handle maximum 5 requests at a time...

Memory javascript dictionary

If I add a value to the 1000th element of a javascript dictionary then is there any difference to adding that value to the 0th element assuming those positions are open. I'm speaking in terms of memory consumption. ex. arr[1000] = 'asdf'; versus arr[0] = 'asdf'; ...

Temporary variable destroyed with overloaded + and += operator

I am getting a segfault because I believe my temporary variable is being destroyed (freeing allocated memory) before the operators are done using it. This is the code in my main: int main(int argc, char *argv[]) { Bcd b1 = Bcd(100), b2 = Bcd(20), b3 = Bcd(50); b3 += b1...

Is there a better way to do this than writing a wrapper allocator that stores a reference to a stateful allocator object?

For example: struct Foo { MyPoolAlloc<char> pool; std::vector<int , MyPoolAlloc<char>> vec_int; // The wrapper allocator would replace MyPoolAlloc<> here. std::vector<std::function<void()> , MyPoolAlloc<char>> vec_funcs; // The wrapper allocator would replace MyPoolAlloc<> here. Foo() : vec_int(pool) , vec_funcs(pool) {} // I want to store lambdas with captured variables using the custom allocator...

Disadvantages of calling realloc in a loop

I'm trying to implement some math algorithms in C on Windows 7, and I need to repeatedly increase size of my array. Sometimes it fails because realloc can't allocate memory. But if I allocate a lot of memory at once in the beginning it works fine. Is it a problem...

Unable to increase PHP Memory Limit

I am running into an issue with a project I am working with on my localhost, where I am unable to increase the PHP memory_limit setting. I've tried increasing it directly in the php.ini config: memory_limit = 1024M I've tried increasing it in the projects .htaccess file: php_value memory_limit 1024M...

fread(), solaris to unix portability and use of uninitialised values

Valgrind found the following error and I, after reading the documentation, the code and other questions in here couldn't figure it out why. Valgrind: first warning ~$ valgrind --vgdb=yes --vgdb-error=0 --read-var-info=yes --leak-check=yes --track-origins=yes debitadmin* debitadmin ==20720== Conditional jump or move depends on uninitialised value(s) ==20720== at 0x4013BC6: initialise (dbg.c:199) ==20720==...