Understanding Address Spaces and the 4GB Limit

Understanding Address Spaces and the 4GB Limit

When the first 32-bit versions of Windows were released back in the early 90's, their large address space was touted as a great feature. At the time, when most machines had less than eight megabytes of memory, a four gigabyte address space seemed incredibly large.

Over the years, advances in semiconductor fabrication technology have made gigabyte memory parts not only a reality, but common place. Desktop machines with a gigabyte of memory common, and boxes configured with four gigabytes of physical memory are not unheard of. These machines are pushing the limits of the system, then, and causing interesting effects of the limitation to be shown.

This article examines the four gigabyte memory addressing limit of Windows 32 operating systems.

Memory Addresses

Memory is the most readily accessible large-volume storage available to the processor. Bytes of data readily accessible to the processor are identified by a memory address, which by convention starts at zero and ranges to the upper limit addressable by the processor. 32-bit processors typically use memory addresses that are 32 bits wide. The 32-bit wide address allows the processor to address 2^32 bytes of memory, which is exactly 4,294,967,296 bytes, or four gigabytes. This is the direct source of the four-gigabyte memory addressing limitation.

Further, all current desktop and server versions of Windows (that is, Windows XP Home, Windows XP Pro, Windows XP Media Center Edition, and all non-embedded editions of Windows 2003 Server, all and editions of Windows 2000) support addressing four gigabytes of physical memory.

Yet accessing four gigabytes of physical memory isn't possible on all machines. Why is that?

While there's no limit in Windows beyond the four-gigabyte limit of the address space, the address space might contain things other than memory. Hardware designers frequently use a technique called memory-mapped I/O. Memory-mapped I/O places memory which is visible to a peripheral on the system bus, in the address space of the processor. To communicate with the device, the processor simply writes data into the range of memory the device has mapped into its address space. This mechanism is significantly faster than using a specialized bus for I/O access, and provides additional benefits the software and hardware designers alike.

Windows will work with the hardware to map as much physical storage memory as possible into the spaces around the device-provided memory mapped features. While the total memory available to the operating system will be less than four gigabytes, the system certainly is capable of accessing four gigabytes of memory space. For applications where memory is important, care must be exercised in selecting and configuring peripherals in order to assure the maximum amount of memory is available.

One easy way to see what memory mapping is happening is to use the Device Manager on your system and have it show what mapping is occurring. On a machine with three gigabytes of memory,

1. Right-click on the "My Computer" icon
2. Select "Properties" from the Context menu
3. In the resulting "System Properties" dialog, activate the "Hardware" tab
4. Click on the "Device Manager" button
5. In the "View" menu, select the "Resources by Connection" command
6. select the "Memory" node in the tree
7. Press the "*" key on the numeric keypad to expand all the nodes in the memory tree

On a desktop machine with three gigabytes of memory, the memory map provided by the Device Manager looks like this:


When this machine, named ENDURO, boots, it counts 3072 megabytes of memory in the BIOS, and shows 3.0 GB in the general properties for the computer. But using the "Performance" tab of Task Manager, this machine reports 3,144,940 kilobytes of available physical memory. This isn't quite enough, as 3072 megabytes of memory is 3,145,728 kilobytes of memory. Where did the missing 788 kilobytes of memory go?

Addresses shown in the Device Manager window are in hexadecimal. We don't care about many of them, but one of the important values we're interested in is 0xC0000000, since that corresponds to three gigabytes. We can see that Windows has allocated addresses 0x00100000 through 0xBFFFFFFF to the system board, and that totals 3,144,704 kilobytes of memory. There's also the range 0x000E0000 through 0x000FFFFF, which is another 128 kilobytes of memory. That brings us within 788 kilobytes of our expected total.

One hole of memory is evident at 0x000A0000 through 0x000BFFFF; this memory is mapped to the display adapter for its text mode and lower graphics modes. These addresses have been used by the dispaly adapter by convention since the original IBM PC in the early eighties, and are retained for compatibility. Adding the memory around it brings us within a few kilobytes of our physical memory goal.

The hole the graphics adapter causes in physical memory is called its aperture. This value is settable in the system BIOS, at the cost of being able to access memory on the video card directly and rapidly. With very large buffers on video cards, and the advent of SLI systems, the memory aperture taken out of the system address space can be huge, and reduces the usable general-purpose memory accessible by the system.

This machine was booted with the BIOS set to a 64MB AGP aperture:



Rebooting the machine and setting the aperture to 256MB makes a noticable difference in the memory map:



While the card still reserves 256 megs for its video memory, the aperture now takes much more room from the system address space. In the first screen shot, we see the aperture is from 0xF8000000 to 0xFBFFFFFF, which is 64 megabytes. On the second screen shot, the aperture extends from 0xE0000000 to 0xEFFFFFFF, which is 256 megabytes.

Had this system four gigs of memory, the system would show at most 3.94 gigabytes of physical memory available after it booted with the 64 megabyte aperture. With the 256 megabyte aperture, the maximum is 3.75 gigabytes. The true maximum is substantially less, since the 256 megabyte card memory also comes from the physical address space, as well as memory for other devices on the system.

Note that all the addresses in this section of the document are physical. The OS is free to map a physical address range to a logical address range in any way it sees fit, and that's important for understanding the next section, where all the addresses described are logical.

The Process Address Space

When a program loads under Windows, it is managed as a process. One of the most important features of a process is its virtual address space, which holds all of the code and data the process will directly access.

In the 32-bit versions of Windows I enumerated above, processes are given a default memory limit of at least two gigabytes. This limit means has the logical addresses that any process touches varying between 0x00000000 and 0x7FFFFFFF. These numbers are interesting because they're the full range of non-negative integers that can fit in 32 bits. Considered as a signed integer, the quantity 0x80000000 is -2147483648,
and the quantity 0xFFFFFFFF is considered -1.

Considering pointers to memory as signed values, even inadvertently, can have disastrous effects. Doing math between two pointers where one is unexpectedly negative results in wildly different results than correctly doing the signed math.

To defend the system against poorly written applications and device drivers, Windows allocates memory for applications from 0x00000000 to 0x7FFFFFFF. It maps the operating system itself in the range 0x80000000 to 0xFFFFFFF; as much as it can, at any rate. This gives user processes two gigabytes of virtual address space, and gives the system two gigabytes of virtual address space.

An application can declare that it has been tested and works correctly with large addresses. Many applications, from Microsoft SQL Server and all its tools, to Adobe Photoshop and Illustrator, have a bit set in their executable image which tells the operating system that the program can load in high address ranges, and be exposed to high memory values—that it doesn't do incorrect signed arithmetic on pointers, and won't get sick if it sees such values.

On a system with more than two gigabytes of memory, the system can be booted with the /3GB option to expose more than two gigabytes of address space to each process that's marked with the large-address aware bit in its executable image. User processes receive three gigabytes of space, then, while the system can use a single gigabyte of address space.

The benefit of using this extra space can be mitigated by the fact that the system has to compress itself into the remaining one gigabyte of address space. Memory-mapped I/O and the video aperture space also come from that single gigabyte. Raymond Chen's excellent Old New Thing blog contains a couple of articles about this tradeoff.


Workarounds and Enhancements

All Pentium-class processors support at least 32-bits of addressing space. Some support more, using a 36-bit address bus. This extension is called PAE, which stands for Physical Address Extensions. A machine with such a processor installed on a motherboard that has a chipset supporting the extensions can use 36 (or more!) bits to address memory. These extra bits are controllable by special registers in the processor and the chipset, and allow the mapping of additional memory into the virtual address space of a process.

Common mostly on server and advanced workstation machines, PAE provides the ability to break through some interesting barriers.

PAE is also used to enable a very interesting feature on some of the newsest processors: data execution protection, or DEP. Data Execution Protection lets the operating system mark ranges of memory as data, which is strictly not executable. Ranges not marked as data are assumed to contain code, which is freely executable. DEP help prevent data from entering the machine and executing as code through buffer overruns, for example.



When DEP is enabled, the system may use /PAE to relocate device drivers above the four gigabyte limit. This enables a simpler tagging mechanism for the data-marked memory to work with PAE, relieving a little bit of address space pressure in the problem. During the development and testing of SP2, it was found that many device drivers didn't work correctly with /PAE enabled when they had to work with DMA transfers that were targeting the address space above four gigs of memory. As such, Windows SP2 turns off PAE and forces the drivers (and possibly their associated mapped regions of memory) into the four gigabyte address space, causing the system to see less available physical memory.

This issue is described in a Microsoft KB article.

Does Win64 Help?

Some people believe that Win64 is of no use to desktop users unless they need to manage monumental amounts of data. This might be true, though the definition of "monumental" might be smaller than we think. With four gigabytes of physical memory, a 32-bit system's physical memory space is rather crowded. Win64 lets the 64-bit processors spread their physical memory out through a much larger logical memory space, resulting in less unusable ranges and fewer conflicts.

Hopefully, vendors will see the light soon and begin focusing on driver support (and stability!) for Win64 operating systems in the desktop and workstation markets.


Disclaimer
This posting is provided "AS IS" with no warranties and confers no rights. I do not speak on behalf of Microsoft. This post contains material protected by copyright.

References

• Intel 440BX Data sheet. (PDF file.) Publication number 29063301, Intel Corporation.
• A description of the 4 GB RAM Tuning feature and the Physical Address Extension switch. Microsoft KB article Q291988.
• Comparison of 32-bit and 64-bit memory architecture for 64-bit editions of Windows XP and Windows Server 2003. Microsoft KB article Q294418.
• Memory Support and Windows Operating Systems, Microsoft WHDC Article.
• Compare the Editions of Windows Server 2003, Microsoft.com article.
• RAM, Virtual Memory, Pagefile and all that stuff, Microsoft support article 555223.
• Windows Internals, Fourth Edition. Russinovitch and Solomon. Microsoft Press, 2004. ISBN 0735619174.
• The amount of RAM reported by the System Properties dialog box and the System Information tool is less than you expect after you install Windows XP Service Pack 2, Microsoft support artcle ID 888137.
• The Old New Thing: The oft-misunderstood /3GB switch, Raymond Chen's MSDN blog.
• The old new Thing: Why can't I see all of the 4GB of RAM in my machine?, Raymond Chen's MSDN blog.
• Changes to Functionality in Microsoft Windows XP Service Pack 2: Part 3: Memory Protection Technologies, a Microsoft TechNet article.
• Operating Systems and PAE Support, an article on the Windows Hardware Developer site.
• Memory Limits for Windows Releases, an article from MSDN.
• 4GB RAM thread on Microsoft's Channel9 site

Version 1.0, 25 March, 2006

• First version.

Version 1.01, 19 July, 2006

• Added some links, fixed some others

Version 1.02, 6 December, 2006

• Added another reference to RayChen's blog

Version 1.03, 19 December, 2006

• Added Channel9 reference.

Version 1.04, 18 February, 2007

• Fix typos.

Version 1.05, 24 August, 2009

• Fix more typos.

for future expansion

Thanks Mike, this was really informative. I already new a fair amount about the 4GB limit, didn't know the arpeture fact though. Also, I remeber hearing that there was a problem with the PCI-Express video cards and Memory on 32-bit systems posted a while back (users would install 3GB of memory, but only 2.5 was being reported). If you have time, could you research that subject and include it in your post?

Never heard of that PCI Express issue before; you'd have to give me a little more to go on. I've only got one PCI Express machine, and it reports 3.0 gigs of memory with 3.0 gigs of memory installed, so I can't reproduce it myself.

Ill try to search the forums, because I know I heard it some where here. Might have been a hardware specific problem then.
Took a look around, seems as though the barrier isn't at 3GB, but 4GB. It may simply have to do with the Apreture you mentioned though, not PCI-Express itself. Some people on various forums were reporting that only 3.5GB (sometimes 3.3GB) was showing up in the system properties, instead of there full 4GB they had installed.

on the A8N SLi Premium , it was a hardware limitation fixed with recent bios releases

nice article , thanks

Originally Posted by Xipher

Some people on various forums were reporting that only 3.5GB (sometimes 3.3GB) was showing up in the system properties, instead of there full 4GB they had installed.

If you install SLI, do you end up using two apertures? Reserving twice the space for video memory? If so, it'll add up quick: two 256 meg cards would take 512 megs out of the address space, plus the aperture space. So that's the 0.50 to 0.75 gigs, right there.

There's plenty of other explanations, though, so it's hard to diagnose without any specifics. I'd just be guessing blindly.

Originally Posted by ThreeDee

on the A8N SLi Premium , it was a hardware limitation fixed with recent bios releases

nice article , thanks

I'm glad you like it. What is the "it" you're referring to?

nice post. did you have to do this for another reason and post it here, or did you just decide to write it for us? either way, thanks.

Very informative Mike. I appreciate you sharing this information and taking the time to type it all out.

Originally Posted by santaliqueur

nice post. did you have to do this for another reason and post it here, or did you just decide to write it for us? either way, thanks.

I wrote it to post here. I see many people who insist that "XP wastes memory", when it doesn't ; I grew weary of responding to them (and getting flamed for taking the host thread off topic), so I put together this essay with the expanation. It's the fourth or fifth such essay I've written and posted here, if I'm counting right.

Thank you for your kind feedback.

I appreciate you sharing this information and taking the time to type it all out.

Yea, thanks. I don't think I would have had the patience to do it.

Can we get a sticky on this?

mikeblas, the link for "RAM, Virtual Memory, Pagefile and all that stuff" is broken.

edit: odoe, thanks for the sticky.

thanks for posting this. It was a good read.

I've updated this article to add a couple interesting links, and fix one that I didn't notice being busted.

That's a good read, thanks mikeblas. I was just given a lecture in another thread about how XP32 had a 2 GB limit. Have yourself a nice cold Moretti for this one.

Originally Posted by Xipher

Thanks Mike, this was really informative. I already new a fair amount about the 4GB limit, didn't know the arpeture fact though. Also, I remeber hearing that there was a problem with the PCI-Express video cards and Memory on 32-bit systems posted a while back (users would install 3GB of memory, but only 2.5 was being reported). If you have time, could you research that subject and include it in your post?

Yeah, I had heard this too, infact on my PC (with 4GB installed, and SLI cards) when I go into my BIOS and "tweek" the PCI-E time, I can get the system to see 2.75GB, but at stock it will only see 2.5GB. I think it's time to get XP 64 on my system.


-Proxy

I've added another, newer reference.

Very nicely done and extremely informative article. Kudos...

Wow.

That is an extremely interesting read. Very informative, especially the PCI-E/SLI information.

I'll definitely be using a 64-bit OS when I start to push the memory envelope though, I'm only 1/2 way there right now.

Thank you for your kind words.

I need to write this up to fit into the article, but I also know one of the guys at work is preparing a KB article to cover the issue. Thing is, X64 only helps if you have a motherboard (eg, a chipset) that supports remapping and a BIOS that allows you to turn it on. All the AMD processors with their on-chip memory controller support it; only certain Intel-compatible chipsets support it.