[Xen-devel] RE: Memory missing in 2.6.32.36

[MaoXiaoyun <tinnycloud@xxxxxxxxxxx>]

Boot up info below.
>From line 4 and 70, see I give dom0 5630M memory,
but from line 230:
229 Memory: 3717504k/52165668k available (4404k kernel code, 1057688k absent, 47389772k reserved, 2847k data, 732k init)

which is printed in  arch/x86/mm/init_64.c, nr_free_pages() only 3717504K,  something wrong?
 
static struct kcore_list kcore_vsyscall;
void __init mem_init(void)
{
 long codesize, reservedpages, datasize, initsize;
 unsigned long absent_pages;
 pci_iommu_alloc();
 /* clear_bss() already clear the empty_zero_page */
 reservedpages = 0;
 /* this will put all low memory onto the freelists */
#ifdef CONFIG_NUMA
 totalram_pages = numa_free_all_bootmem();
#else
 totalram_pages = free_all_bootmem();
#endif
 absent_pages = absent_pages_in_range(0, max_pfn);
 reservedpages = max_pfn - totalram_pages - absent_pages;
 after_bootmem = 1;
 codesize =  (unsigned long) &_etext - (unsigned long) &_text;
 datasize =  (unsigned long) &_edata - (unsigned long) &_etext;
 initsize =  (unsigned long) &__init_end - (unsigned long) &__init_begin;
 /* Register memory areas for /proc/kcore */
 kclist_add(&kcore_vsyscall, (void *)VSYSCALL_START,
    VSYSCALL_END - VSYSCALL_START, KCORE_OTHER);
 printk(KERN_INFO "Memory: %luk/%luk available (%ldk kernel code, "
    "%ldk absent, %ldk reserved, %ldk data, %ldk init)\n",
  nr_free_pages() << (PAGE_SHIFT-10),
  max_pfn << (PAGE_SHIFT-10),
  codesize >> 10,
  absent_pages << (PAGE_SHIFT-10),
  reservedpages << (PAGE_SHIFT-10),
  datasize >> 10,
  initsize >> 10);
}
 
  1 root (hd0,0)
  2  Filesystem type is ext2fs, partition type 0x83
  3 kernel /xen-4.0.1.gz msi=1 iommu=off x2apic=off console=com1,vga com1=115200,8n
  4 1 noreboot dom0_mem=5630M dom0_max_vcpus=4 dom0_vcpus_pin cpuidle=0 cpufreq=none
  5                                                                     
  6  \ \/ /___ _ __   | || |  / _ \ / |                                     &n bsp;  *
  7   \  // _ \ '_ \  | || |_| | | || |                                        *
  8   /  \  __/ | | | |__   _| |_| || |                                        *
  9  /_/\_\___|_| |_|    |_|(_)___(_)_|                                        *
 10 (XEN) ACPI: RSDP 000FA000, 0024 (r2 ACPIAM)
 11 (XEN) ACPI: XSDT BF790100, 005C (r1 080510 XSDT1021 20100805 MSFT       97)
 12 (XEN) ACPI: FACP BF790290, 00F4 (r4 080510 FACP1021 20100805 MSFT       97)
 13 (XEN) ACPI: DSDT BF7904F0, 4E48 (r2  CTSAV CTSAV031       31 INTL 20051117)
 14 (XEN) ACPI: FACS BF79E000, 0040
 15 (XEN) ACPI: APIC BF790390, 0118 (r2 080510 APIC1021 20100805 MSFT       97)
 16 (XEN) ACPI: MCFG BF7904B0, 003C (r1 080510 OEMMCFG  20100805 MSFT       97)
 17 (XEN) ACPI: OEMB BF79E040, 0082 (r1 080510 OEMB1021 20100805 MSFT       97)
 18 (XEN) ACPI: SRAT BF79A4F0, 01D0 (r2 080510 OEMSRAT         1 INTL        1)
 19 (XEN) ACPI: HPET BF79A6C0, 0038 (r1 080510 OEMHPET&n bsp; 20100805 MSFT       97)
 20 (XEN) ACPI: SSDT BF7A1A10, 0363 (r1 DpgPmm    CpuPm       12 INTL 20051117)
 21 (XEN) System RAM: 49143MB (50322600kB)
 22 (XEN) Domain heap initialised DMA width 32 bits
 23 (XEN) Processor #0 6:12 APIC version 21
 24 (XEN) Processor #2 6:12 APIC version 21
 25 (XEN) Processor #18 6:12 APIC version 21
 26 (XEN) Processor #20 6:12 APIC version 21
 27 (XEN) Processor #32 6:12 APIC version 21
 28 (XEN) Processor #34 6:12 APIC version 21
 29 (XEN) Processor #50 6:12 APIC version 21
 30 (XEN) Processor #52 6:12 APIC version 21
 31 (XEN) Processor #1 6:12 APIC version 21
 32 (XEN) Processor #3 6:12 APIC version 21
 33 (XEN) Processor #19 6:12 APIC version 21
 
 66 (XEN) *** LOADING DOMAIN 0 ***
 67 (XEN)  Xen  kernel: 64-bit, lsb, compat32
 68 (XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x19b7000
 69 (XEN) PHYSICAL MEMORY ARRANGEMENT:
 70 (XEN)  Dom0 alloc.:   0000000434000000->0000000438000000 (1424896 pages to be allocated)
 71 (XEN) VIRTUAL MEMORY ARRANGEMENT:
 72 (XEN)  Loaded kernel: ffffffff81000000->ffffffff819b7000
 73 (XEN)  Init. ramdisk: ffffffff819b7000->ffffffff81ec6e00
 74 (XEN)  Phys-Mach map: ffffffff81ec7000->ffffffff829c6000
 75 (XEN)  Start info:    ffffffff829c6000->ffffffff829c64b4
 76 (XEN)  Page tables:   ffffffff829c7000->ffffffff829e0000
 77 (XEN)  Boot stack:    ffffffff829e0000->ffffffff829e1000
 78 (XEN)  TOTAL:         ffffff ff80000000->ffffffff82c00000
 79 (XEN)  ENTRY ADDRESS: ffffffff8172c200
 80 (XEN) Dom0 has maximum 4 VCPUs
 81 (XEN) Scrubbing Free RAM: ...............................................................................................................................    .........................................................................................................................................................    ..............
 82 ...............................
 83 (XEN) Xen trace buffers: disabled
 84 (XEN) Std. Loglevel: Errors and warnings
 85 (XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
 86 (XEN) Xen is relinquishing VGA console.
 87 (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
 88 (XEN) Freed 168kB init memory.
 89 mapping kernel into physical memory
 90 Xen: setup ISA identity maps
 91 about to get started...
 92 Initializing cgroup subsys cpuset
 93 Initializing cgroup subsys cpu
 94 Linux version 2.6.32.36xen (root@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Wed May 18 22:10:07 CST 2011
 95 Command line: ro root=LABEL=/ hda=noprobe console=hvc0
 96 KERNEL supported cpus:
 97   Intel GenuineIntel
 98   AMD AuthenticAMD
&nb sp;99   Centaur CentaurHauls
100 xen_release_chunk: looking at area pfn bf7e0-bf7ec: 12 pages freed
101 xen_release_chunk: looking at area pfn c0000-e0000: 131072 pages freed
102 xen_release_chunk: looking at area pfn f0000-fec00: 60416 pages freed
103 xen_release_chunk: looking at area pfn fec01-fec8a: 137 pages freed
104 xen_release_chunk: looking at area pfn fec8b-fee00: 373 pages freed
105 xen_release_chunk: looking at area pfn fee01-fff00: 4351 pages freed
106 released 196361 pages of unused memory
107 BIOS-provided physical RAM map:
108  Xen: 0000000000000000 - 000000000009a000 (usable)
109  Xen: 000000000009a400 - 0000000000100000 (reserved)
110  Xen: 0000000000100000 - 00000000bf790000 (usable)
111  Xen: 00000000bf790000 - 00000000bf79e000 (ACPI data)
112  Xen: 00000000bf79e000 - 00000000bf7d0000 (ACPI NVS)
113  Xen: 00000000bf7d0000 - 00000000bf7e0000 (reserved)
114  Xen: 000000 00bf7ec000 - 00000000c0000000 (reserved)
115  Xen: 00000000e0000000 - 00000000f0000000 (reserved)
116  Xen: 00000000fec00000 - 00000000fec01000 (reserved)
117  Xen: 00000000fec8a000 - 00000000fec8b000 (reserved)
118  Xen: 00000000fee00000 - 00000000fee01000 (reserved)
119  Xen: 00000000fff00000 - 0000000100000000 (reserved)
120  Xen: 0000000100000000 - 0000000c6ff09000 (usable)
121 DMI present.
122 AMI BIOS detected: BIOS may corrupt low RAM, working around it.
123 last_pfn = 0xc6ff09 max_arch_pfn = 0x400000000
124 last_pfn = 0xbf790 max_arch_pfn = 0x400000000
125 init_memory_mapping: 0000000000000000-00000000bf790000
126 init_memory_mapping: 0000000100000000-0000000c6ff09000
127 RAMDISK: 019b7000 - 01ec6e00
128 ACPI: RSDP 00000000000fa000 00024 (v02 ACPIAM)
129 ACPI: XSDT 00000000bf790100 0005C (v01 080510 XSDT1021 20100805 MSFT 00000097)
130 ACPI: FACP 00000000bf790290 000F4 (v04 080510 FACP1021 20100805 MSFT 00000097)
131 ACPI: DSDT 00000000bf7904f0 04E48 (v02  CTSAV CTSAV031 00000031 INTL 20051 117)
132 ACPI: FACS 00000000bf79e000 00040
133 ACPI: APIC 00000000bf790390 00118 (v02 080510 APIC1021 20100805 MSFT 00000097)
134 ACPI: MCFG 00000000bf7904b0 0003C (v01 080510 OEMMCFG  20100805 MSFT 00000097)
135 ACPI: OEMB 00000000bf79e040 00082 (v01 080510 OEMB1021 20100805 MSFT 00000097)
136 ACPI: SRAT 00000000bf79a4f0 001D0 (v02 080510 OEMSRAT  00000001 INTL 00000001)
137 ACPI: HPET 00000000bf79a6c0 00038 (v01 080510 OEMHPET  20100805 MSFT 00000097)
138 ACPI: SSDT 00000000bf7a1a10 00363 (v01 DpgPmm    CpuPm 00000012 INTL 20051117)
139 (10 early reservations) ==> bootmem [0000000000 - 0c6ff09000]
140   #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
141   #1 [00029c7000 - 00029e0000]   XEN PAGETABLES ==> [00029c7000 - 00029e0000]
142   #2 [0000006000 - 0000008000]       TRAMPOLINE ==> [0000006 000 - 0000008000]
143   #3 [0001000000 - 000188c3b8]    TEXT DATA BSS ==> [0001000000 - 000188c3b8]
144   #4 [00019b7000 - 0001ec6e00]          RAMDISK ==> [00019b7000 - 0001ec6e00]
145   #5 [0001ec7000 - 00029c7000]   XEN START INFO ==> [0001ec7000 - 00029c7000]
146   #6 [015fe00000 - 0c6ff09000]        XEN EXTRA ==> [015fe00000 - 0c6ff09000]
147   #7 [000188d000 - 00018a31c0]              BRK ==> [000188d000 - 00018a31c0]
148   #8 [0000100000 - 00006e4000]          PGTABLE ==> [0000100000 - 00006e4000]
149   #9 [00029e0000 - 000858e000]          PGTABLE ==> [00029e0000 - 000858e000]
150 Zone PFN ranges:
151   DMA      0x00000010 -> 0x00001000
152   DMA32    0x00001000 -> 0x00100000
153   Normal   0x00100000 -> 0x00c6ff09
154 Movable zone start PFN for each node
155 early_node_map[3] active PFN ranges
156     0: 0x00000010 -> 0x0000009a
157     0: 0x00000100 -> 0x000bf790
158     0: 0x00100000 -> 0x00c6ff09
159 ACPI: PM-Timer IO Port: 0x808
160 ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
161 ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
162 ACPI: LAPIC (acpi_id[0x03] lapic_id[0x12] enabled)
163 ACPI: LAPIC (acpi_id[0x04] lapic_id[0x14] enabled)
164 ACPI: LAPIC (acpi_id[0x05] lapic_id[0x20] enabled)
165 ACPI: LAPIC (acpi_id[0x06] lapic_id[0x22] enabled)
166 ACPI: LAPIC (acpi_id[0x07] lapic_id[0x32] enabled)
167 ACPI: LAPIC (acpi_id[0x08] l apic_id[0x34] enabled)
168 ACPI: LAPIC (acpi_id[0x09] lapic_id[0x01] enabled)
169 ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x03] enabled)
170 ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x13] enabled)
171 ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x15] enabled)
172 ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x21] enabled)
173 ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x23] enabled)
174 ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x33] enabled)
175 ACPI: LAPIC (acpi_id[0x10] lapic_id[0x35] enabled)
176 ACPI: LAPIC (acpi_id[0x11] lapic_id[0x90] disabled)
177 ACPI: LAPIC (acpi_id[0x12] lapic_id[0x91] disabled)
178 ACPI: LAPIC (acpi_id[0x13] lapic_id[0x92] disabled)
179 ACPI: LAPIC (acpi_id[0x14] lapic_id[0x93] disabled)
180 ACPI: LAPIC (acpi_id[0x15] lapic_id[0x94] disabled)
181 ACPI: LAPIC (acpi_id[0x16] lapic_id[0x95] disabled)
182 ACPI: LAPIC (acpi_id[0x17] lapic_id[0x96] disabled)
183 ACPI: LAPIC (acpi_id[0x18] lapic_id[0x97] disabled)
184 ACPI: IOAPIC (id[0x06] address[0xfec00000] gsi_base[0])
185 IOAPIC[0]: apic_id 6, version 0, address 0xfec00000, GSI 0-0
186 ACPI: IOAPIC (id[0x07] address[0xfec8a000] gsi_base[24])
187 IOAPIC[1]: apic_id 7, version 0, address 0xfec8a000, GSI 24-24
188 ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
189 ERROR: Unable to locate IOAPIC for GSI 2
190 ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
191 ERROR: Unable to locate IOAPIC for GSI 9
192 Using ACPI (MADT) for SMP configuration information
193 ACPI: HPET id: 0x8086a301 base: 0xfed00000
194 SMP: Allowing 4 CPUs, 0 hotplug CPUs
195 PM: Registered nosave memory: 000000000009a000 - 000000000009b000
196 PM: Registered nosave memory: 000000000009b000 - 0000000000100000
197 PM: Registered nosave memory: 00000000bf790000 - 00000000bf79e000
198 PM: Registered nosave memory: 00000000bf79e000 - 00000000bf7d0000
199 PM: Registered nosave memory: 00000000bf7d0000 - 00000000bf7e0000
200 PM: Registered nosave memory: 00000000bf7e0000 - 00000000bf7ec000
201 PM: Registered nosave memory: 00000000bf7ec000 - 00000000c0000000
202 PM: Registered nosave memory: 00000000c0000000 - 00000000e0000000
203 PM: Registered nosave memory: 00000000e0000000 - 00000000f0000000
204 PM: Registered nosave memory: 00000000f0000000 - 00000000fec00000
205 PM: Registered nosave memory: 00000000fec00000 - 00000000fec01000
206 PM: Registered nosave memory: 00000000fec01000 - 00000000fec8a000
207 PM: Registered nosave memory: 00000000fec8a000 - 00000000fec8b000
208 PM: Registered nosave memory: 00000000fec8b000 - 00000000fee00000
209 PM: Registered nosave memory: 00000000fee00000 - 00000000fee01000
210 PM: Registered nosave memory: 00000000fee01000 - 00000000fff00000
211 PM: Registered nosave memory: 00000000fff00000 - 0000000100000000
212 Allocating PCI resources starting at c0000000 (gap: c0000000:20000000)
213 Booting para virtualized kernel on Xen
214 Xen version: 4.0.1 (preserve-AD) (dom0)
215 NR_CPUS:64 nr_cpumask_bits:64 nr_cpu_ids:4 nr_node_ids:1
216 PERCPU: Embedded 29 pages/cpu @ffff88002803b000 s90072 r8192 d20520 u118784
217 pcpu-alloc: s90072 r8192 d20520 u118784 alloc=29*4096
218 pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3
219 Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 12597183
220 Kernel command line: ro root=LABEL=/ hda=noprobe console=hvc0
221 PID hash table entries: 4096 (order: 3, 32768 bytes)
222 Dentry cache hash table entries: 8388608 (order: 14, 67108864 bytes)
223 Inode-cache hash table entries: 4194304 (order: 13, 33554432 bytes)
224 Initializing CPU#0
225 DMA: Placing 64MB software IO TLB between ffff880020000000 - ffff880024000000
226 DMA: software IO TLB at phys 0x20000000 - 0x24000000
227 xen_swiotlb_fixup: buf=ffff880020000000 size=67108864
228 xen_swiotlb_fixup: buf=ffff880024060000 size=32768
229 Memory: 3717504k/52165668k available (4404k kernel code, 1057688k absent, 47389772k reserved, 2847k data, 732k init)
230 SLUB: Genslabs=13, HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
231 Hierarchical RCU implementation.
232 NR_IRQS:4352 nr_irqs:1280
233 ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
234 ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
235 xen: sci override: source_irq=9 global_irq=9 trigger=c polarity=1
236 xen_allocate_pirq: returning irq 9 for gsi 9
237 xen: acpi sci 9
238 Console: colour VGA+ 80x25
239 console [hvc0] enabled
240 allocated 511180800 bytes of page_cgroup
241 please try 'cgroup_disable=memory' option if you don't want memory cgroups
242 installing Xen timer for CPU 0

---

From: tinnycloud@xxxxxxxxxxx
To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: Memory missing in 2.6.32.36
Date: Thu, 19 May 2011 11:58:01 +0800

if xend is not started at the very beginning
 
root@xmao # free -m
             total       used       free     shared    buffers     cached
Mem:          3636       1020       2616          0         12         52
-/+ buffers/cache:        955       2681
Swap:         1027          0       1027
 

---

From: tinnycloud@xxxxxxxxxxx
To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Memory missing in 2.6.32.36
Date: Wed, 18 May 2011 22:58:55 +0800

Hi:
 
    Recently I am trying pvops dom0 kernel, 2.6.32.36, but confronted some strange things.
 
    In my grub, i give dom0 5630M memory. But when system startup, use free -m, only see 2869M in total.
 
title Red Hat Enterprise Linux Server (2.6.32.36xen)
        root (hd0,0)
        kernel /xen-4.0.1.gz msi=1 iommu=off x2apic=off console=com1,vga com1=115200,8n1 noreboot dom0_mem=5630M dom0_max_vcpus=4 dom0_vcpus_pin cpuidle=    0 cpufreq=none no-xsave
        module /vmlinuz-2.6.32.36xen ro root=LABEL=/ hda=noprobe console=hvc0
        module /initrd-2.6.32.36xen.img
 
 
Free -m output
 
root@xmao # free -m
             to tal       used       free     shared    buffers     cached
Mem:          2869       1068       1800          0         25         65
-/+ buffers/cache:        977       1892
Swap:         1027          0       1027
 
root@xmao# xm li
Name                            &nbs p;           ID   Mem VCPUs      State   Time(s)
Domain-0                                     0  4862     4     r-----   1272.6
 
root@xmao # xenstore-read /local/domain/0/memory/target
4979676
 
and dom0 auto balloon is disabled in /etc/xen/xend-config.
 
Well, when I use 2.6.32.10. Everything is OK
Free output in 2.6.32.10
root@xmao # free -m
             total       used       free     shared     buffers     cached
Mem:          4860       1550       3309          0         14         75
-/+ buffers/cache:       1460       3399
Swap:         1027          0       1027
 
Where is my memory going with 32.36?
 
Thanks.
 
 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel