Page Modification Logging for Virtual Machine Monitor White Paper

Citation preview

Page Modification Logging for Virtual Machine Monitor White Paper This document is intended only for VMM or hypervisor software developers and not for application developers or end-customers. Readers are expected to be knowledgeable about Intel® Architecture and Intel® Virtualization Technology.

331872-001 January 2015

PAGE MODIFICATION LOGGING FOR VIRTUAL MACHINE MONITOR

I nt el t echnologies feat ures and benefit s depend on sy st em configurat ion and m ay r equire enabled har dwar e, soft war e, or ser vice act ivat ion. Lear n m or e at int el.com , or fr om t he OEM or r et ailer. No com put er sy st em can be absolut ely secur e. I nt el does not assum e any liabilit y for lost or st olen dat a or sy st em s or any dam ages result ing fr om such losses. You m ay not use or facilit at e t he use of t his docum ent in connect ion w it h any infringem ent or ot her legal analysis concer ning I nt el product s descr ibed her ein. You agr ee t o grant I nt el a non- exclusive, r oyalt y- fr ee license t o any pat ent claim t hereaft er draft ed w hich includes subj ect m at t er disclosed her ein. No license ( ex pr ess or im plied, by est oppel or ot herw ise) t o any int ellect ual propert y r ight s is grant ed by t his docum ent . The pr oduct s descr ibed m ay cont ain design defect s or er rors k now n as er rat a w hich m ay cause t he product t o dev iat e from published specificat ions. Curr ent charact erized er rat a are available on request . This docum ent cont ains inform at ion on pr oduct s, serv ices and/ or pr ocesses in developm ent . All inform at ion prov ided her e is subj ect t o change w it hout not ice. Cont act your I nt el repr esent at ive t o obt ain t he lat est I nt el pr oduct specificat ions and r oadm aps Copies of docum ent s w hich have an order num ber and are referenced in t his docum ent , or ot her I nt el lit erat ur e, m ay be obt ained by calling 1- 800- 548- 4725, or by v isit ing ht t p: / / w w w.int el.com / design/ lit erat ure.ht m . I nt el, t he I nt el logo, I nt el At om , I nt el Cor e, I nt el SpeedSt ep, MMX, Pent ium , VTune, and Xeon are t radem ar k s of I nt el Cor porat ion in t he U.S. and/ or ot her count ries. * Ot her nam es and brands m ay be claim ed as t he pr oper t y of ot her s.

Copy r ight © 2015, I nt el Cor porat ion. All Right s Reser ved.

January 2015

331872-001

ii

PAGE MODIFICATION LOGGING FOR VIRTUAL MACHINE MONITOR

CHAPTER 1 PAGE MODIFICATION LOGGING 1.1 OVERVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 VMCS CHANGES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 CHANGES TO VMX NON-ROOT OPERATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 CHANGES TO VM ENTRIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 CHANGES TO VM EXITS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.6 CHANGES TO VMX CAPABILITY REPORTING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

January 2015

331872-001

1-1 1-2 1-2 1-3 1-4 1-4

iii

PAGE MODIFICATION LOGGING FOR VIRTUAL MACHINE MONITOR

CHAPTER 1 PAGE MODIFICATION LOGGING 1.1

OVERVIEW

This paper describes an I nt el ® Virt ualizat ion Technology ( I nt el ® VT) enhancem ent for fut ure I nt el processors. This feat ure, referred t o as pa ge - m odifica t ion logging ( PM L) , furt her ext ends t he capabilit y of virt ual- m achine m onit or ( VMM) soft ware t hat em ploys t he ext ended page- t able m echanism ( EPT) by allowing t hat soft ware t o m onit or t he guest- physical pages m odified during guest virt ual m achine ( VM) execut ion m ore efficient ly. Det ails of I nt el VT, including EPT, can be found in I nt el ® 64 and I A- 32 Archit ect ures Soft ware Developer’s Manual ( SDM) in Volum e 3C. Earlier I nt el processors int roduced accessed and dirt y flags for EPT; see Sect ion 28.2.4, “Accessed and Dirt y Flags for EPT,” of I nt el ® 64 and I A- 32 Archit ect ures Soft ware Developer’s Manual ( SDM) in Volum e 3C. This feat ure enables VMMs t o im plem ent m em ory- m anagem ent and page- classificat ion algorit hm s efficient ly so as t o opt im ize VM m em ory operat ions such as defragm ent at ion, paging, et c. Wit hout accessed and dirt y flags, VMMs m ay need t o em ulat e t hem ( by m arking EPT pagingst ruct ures as not- present or read- only) and incur t he overhead of t he result ing EPT violat ions: VM exit s and associat ed soft ware processing. For som e usage m odels, VMMs m ay benefit from addit ional support t o m onit or t he working set size. As accessed and dirt y flags for EPT are set wit hout invoking t he VMM, t here is no opport unit y for t he VMM t o accum ulat e working- set st at ist ics during operat ion. To calculat e such st at ist ics t he VMM m ust scan t he EPT paging st ruct ures t o aggregat e t he inform at ion from t he accessed and dirt y flags. Such scans im pose bot h lat ency and bandwidt h cost s t hat m ay be unaccept able in som e circum st ances. PML builds upon t he processor ’s support for accessed and dirt y flags for EPT, ext ending t he processing t hat occurs when dirt y flags for EPT are set . When PML is act ive each writ e t hat set s a dirt y flag for EPT also generat es an ent ry in an inm em ory log, report ing t he guest- physical address of t he writ e ( aligned t o 4 KByt es) . When t he log is full, a VM exit occurs, not ifying t he VMM. A VMM can m onit or t he num ber of pages m odified by each t hread by specifying an available set of log ent ries. The rest of t his paper is organized in subsect ions which cover specific changes in VMX t o support PML:

• • • • •

Sect ion 1.2 det ails changes t o t he VMCS int roduced wit h t he PML feat ure. Sect ion 1.3 det ails changes t o VMX non- root operat ion. Sect ion 1.4 describes changes t o VM ent ries. Sect ion 1.5 describes changes t o VM exit s. Sect ion 1.6 det ails changes t o t he capabilit y report ing int roduced wit h PML.

January 2015

331872-001

1-1

PAGE MODIFICATION LOGGING FOR VIRTUAL MACHINE MONITOR

1.2

VMCS CHANGES

Secondary processor- based VM- execut ion cont rol 17 is defined as e na ble PM L. A new 64- bit VM- execut ion cont rol field is defined called t he PM L a ddr e ss. This is t he 4- KByt e aligned physical address of t he pa ge - m odifica t ion log. The pagem odificat ion log com prises 512 64- bit ent ries. The VMCS- field encoding pair for t he PML address is 0000200EH ( for all 64 bit s) and 0000200FH ( for t he upper 32 bit s) . A new 16- bit guest- st at e field is defined called t he PM L in de x . The PML index is t he logical index of t he next ent ry in t he page m odificat ion log. Because t he page- m odificat ion log com prises 512 ent ries ( see above) , t he PML index is t ypically a value in t he range 0–511. The VMCS- field encoding for t he PML index is 00000812H. The PML address and PML index fields exist only on processors t hat support t he 1set t ing of t he “ enable PML” VM- execut ion cont rol.

1.3

CHANGES TO VMX NON-ROOT OPERATION

I f t he “ enable PML” VM- execut ion cont rol is 1 and bit 6 of EPT point er ( EPTP) is 1 ( enabling accessed and dirt y flags for EPT) , t he behavior in VMX non- root operat ion is m odified as described in t his sect ion 1 . Before allowing a guest- physical access, t he processor m ay det erm ine t hat it first needs t o set an accessed or dirt y flag for EPT. When t his happens, t he processor exam ines t he PML index. I f t he PML index is not in t he range 0–511 ( bit s 15: 9 of PML index are not all 0) , a VM exit occurs. The accessed or dirt y flag is not set , and t he guest- physical access t hat t riggered t he event does not occur. See Sect ion 1.5 for det ails of how t he VM exit occurs. I f inst ead t he PML index is in t he range 0–511 and a guest- physical access causes t he processor t o updat e a dirt y flag for EPT ( changing it from 0 t o 1) , t he processor operat es as follows:

•

The guest- physical address of t he access is writ t en t o t he page- m odificat ion log. Specifically, t he guest- physical address is writ t en t o physical address det erm ined by adding 8 t im es t he PML index t o t he PML address. Bit s 11: 0 of t he value writ t en are always 0 ( t he guest- physical address writ t en is 4- KByt e aligned) .

•

The PML index is decrem ent ed by 1 ( t his m ay cause t he value t o t ransit ion from 0 t o FFFFH) .

Because t he processor decrem ent s t he PML index wit h each log ent ry, t he value will event ually wrap around t o FFFFH. No furt her logging will occur because, t he next t im e a log ent ry is required, t he processor will det erm ine t hat bit s 15: 9 of t he PML index are not all 0 and will cause a VM exit ( see above and Sect ion 1.5) .

1. If bit 6 of EPT pointer (EPTP) is 0 (disabling accessed and dirty flags for EPT), the setting of the “enable PML” VM-execution control has no effect on VMX non-root operation.

1-2

331872-001

January 2015

PAGE MODIFICATION LOGGING FOR VIRTUAL MACHINE MONITOR

The following pseudocode com prehends t he im pact of t he 1- set t ing of “ enable PML” on an updat e t o an accessed or dirt y flag for EPT: IF (PML Index[15:9] ≠ 0) THEN VM exit; FI; set accessed and dirty flags for EPT; IF (a dirty flag was updated from 0 to 1) THEN PML address[PML index] ← 4-KByte-aligned guest-physical address; PML index is decremented; FI;

1.4

CHANGES TO VM ENTRIES

I f t he “ act ivat e secondary cont rols” and “ enable PML” VM- execut ion cont rols are bot h 1, VM ent ries ensure t he following:

• • •

The “ enable EPT” VM- execut ion cont rol is 1. Bit s 11: 0 of t he PML address are 0. The PML address does not set any bit s beyond t he processor ’s physical- address widt h. 1

VM ent ry fails if t his check fails. When such a failure occurs, cont rol passes t o t he next inst ruct ion, RFLAGS.ZF is set t o 1 t o indicat e t he failure, and t he VM- inst ruct ion error field is loaded wit h value 7, indicat ing “ VM ent ry wit h invalid cont rol field( s) .” This check m ay be perform ed in any order wit h respect t o ot her checks on VMX cont rols and t he host- st at e area. Different processors m ay t hus give different error num bers for t he sam e VMCS. The “ enable PML” VM- execut ion cont rol m ay be 1 even if bit 6 of EPT point er ( EPTP) is 0 ( disabling accessed and dirt y flags for EPT) ; VM ent ry will not fail because of t his condit ion. As not ed in foot not e 1 in Sect ion 1.3, t he set t ing of t he “ enable PML” VMexecut ion cont rol has no effect on VMX non- root operat ion if accessed and dirt y flags for EPT are disabled. VM ent ry does not check t he value of t he PML index, and t hat value will not cause VM ent ry t o fail, even if bit s 15: 9 of t he value are not all 0.

1. Software can determine a processor’s physical-address width by executing CPUID with 80000008H in EAX. The physical-address width is returned in bits 7:0 of EAX.

January 2015

331872-001

1-3

PAGE MODIFICATION LOGGING FOR VIRTUAL MACHINE MONITOR

1.5

CHANGES TO VM EXITS

As indicat ed in Sect ion 1.3, t he processor causes a VM exit when t he next PML index specifies an invalid locat ion ( because bit s 15: 9 of PML index are not all 0) . Typically, t his will occur when t he page- m odificat ion log is full and t he PML index has wrapped around from 0 t o FFFFH. VM exit s result ing from t he value of t he PML index use a basic exit reason of 62 ( 3EH) , indicat ing “ page- m odificat ion log full.” VM exit s due t o “ page- m odificat ion log full” do save an exit qualificat ion defined as follows:

• •

Bit s 11: 0 are undefined. Bit 12 is undefined in eit her of t he following cases: — I f t he “ NMI exit ing” VM- execut ion cont rol is 1 and t he “ virt ual NMI s” VMexecut ion cont rol is 0. — I f t he VM exit set s t he valid bit in t he I DT- vect oring inform at ion field. Ot herwise, bit 12 is defined as follows: — I f t he “ virt ual NMI s” VM- execut ion cont rol is 0, t he EPT violat ion was caused by a m em ory access as part of execut ion of t he I RET inst ruct ion, and blocking by NMI was in effect before execut ion of I RET, bit 12 is set t o 1. — I f t he “ virt ual NMI s” VM- execut ion cont rol is 1,t he EPT violat ion was caused by a m em ory access as part of execut ion of t he I RET inst ruct ion, and virt ualNMI blocking was in effect before execut ion of I RET, bit 12 is set t o 1. — For all ot her relevant VM exit s, bit 12 is cleared t o 0.

•

Bit s 63: 13 are undefined.

These VM exit s save t he I DT- vect oring inform at ion and I DT- vect oring error code as t hey are for VM exit s due t o EPT violat ions. Thus, if t he VM exit occurred while delivering an event t hrough t he I DT, t hese fields receive inform at ion about t hat event .

1.6

CHANGES TO VMX CAPABILITY REPORTING

Sect ion 1.3 specified t hat bit 17 of t he secondary processor- based VM- execut ion cont rols is defined as “ enable PML”. A processor t hat support s t he 1- set t ing of “ enable PML” set s bit 49 of t he I A32_VMX_PROCBASED_CTLS2 MSR ( index 48BH) . RDMSR of t hat MSR ret urns 1 in bit 17 of EDX.

1-4

331872-001

January 2015