genivi-projects@lists.genivi.org

Technical discussion on GENIVI projects

View all threads

[Perf Tooling] Presentation of Emma Memory and Mapfile Analyser

MS
Marcel.Schmalzl@partner.bmw.de
Mon, Jan 25, 2021 12:05 PM

Dear all,

If you are interested in performance values of your embedded device / ECU this project could be of your interest. For some time now we were developing the Emma Memory and Mapfile Analyser (Emma)https://github.com/bmwcarit/Emma/ which helps us and suppliers as a common base for static memory consumption analyses based on map file input. To get a first impression you can test it yourself by running the test projecthttps://github.com/bmwcarit/Emma/tree/master/doc/test_project which you can find in the repository.

Our main motivations:

  •     Having an output that is
    

o  machine readable for further processing/evaluation,

o  easy to navigate/filter from a developer's perspective and

o  presentable to a non-technical audience.

Features

?  Memory analysis for systems ranging from simple bare-metal microcontroller projects to complex projects involving multiple SoCs

?  Plausibility checks of memory entries:

o  E.g. sections/objects outside physical memory

o  Labelling and handling of overlap, containment, duplicate scenarios

?  Support for virtual to physical memory address translation (see "Limitations" below)

?  Two types of memory region abstractions:

o  1st level is user defined (e.g. SDRAM, CCM_RAM, ...)

o  2nd level is fixed to: INT_RAM, INT_ROM, EXT_RAM, EXT_ROM)

?  Custom section and object categorisation
(based on architecture, taste, e.g. GUI, Tier1, OEM; see "Emma processing steps" below)

?  Creating different summaries (see figure below):

o  Section_Summary: Contains the resolved* section data

o  Object_Summary: Contains the resolved* object data

o  Objects_in_Sections: Contains the merged section and object summaries

  • To avoid counting the same memory regions multiple times. The scenarios are labelled as overlap, containment or duplication.

[objectsInSectionsResolution.png]

?  Processing of GreenHills or similarly formatted map files
(see also "Upcoming features" section below)

?  User guidance through interactive command line

?  Options for seamless integration in CI systems (--noprompt, -Werror)

?  Teamscalehttps://www.cqse.eu/en/teamscale/overview/ support (via non-code metrics)

?  Extensive documentation and examples for a quick start

Limitations

  •     Emma requires devices to have a single address space (e.g. ARMv6M):
    

o  Some architectures have separate code and data address spaces that both start at address 0x00000000 (e.g. Intel 8051). Those architectures cannot entirely be analysed with Emma.

  •     The stacks and heaps need to be statically defined
    
  •     The virtual address spaces (VAS) need to be statically defined
    

o  Devices using memory management units (MMU) are not supported

o  This means that systems running complex OSs (e.g. Linux, Windows) are not supported*

The Emma components and the dataflow between them

It is a Python-based (pip install pypiemma) tool-chain which consists of three modules:

  •     Emma (analyser):  the main executable you start with, it creates .csv data from the map files
    
  •     Emma Visualiser: creates visualisations and displays consumption from the .csv data
    
  •     Emma Deltas: compare/diff analysis results
    

[workflow.png]

Emma processing steps

Step 1: As a common use case you would start with map files consisting of physical and/or virtual addresses. Emma will map all virtual addresses back to physical addresses using an address translation map file.

Step 2: For each section and object it will be checked to which memory region it belongs.

Step 3: Based on the category configuration each and every section/object is mapped to a category.

After that you can start Emma Visualiser which will aggregate the .csv entries and creates automated reports and figures.

[cid:image003.jpg@01D6F31A.CDCF1C10]

Upcoming features

?  Support for new map file formats, like GCC

?  Visualisation of objects and sections for a selected memory region as .svg. -> arrives end of January 2021
Example for a simple scenario:

[cid:image004.png@01D6F31A.CDCF1C10]

Example for a complex scenario:

[test6]

I hope you found this interesting and Emma might help you facilitating your daily work. If you have further questions, feel free to reach out to us via your mailing list (emma-dev (. at) googlegroups.com).

Best regards,

Marcel Schmalzl and the Emma development team.

Dear all, If you are interested in performance values of your embedded device / ECU this project could be of your interest. For some time now we were developing the Emma Memory and Mapfile Analyser (Emma)<https://github.com/bmwcarit/Emma/> which helps us and suppliers as a common base for static memory consumption analyses based on map file input. To get a first impression you can test it yourself by running the test project<https://github.com/bmwcarit/Emma/tree/master/doc/test_project> which you can find in the repository. Our main motivations: * Having an output that is o machine readable for further processing/evaluation, o easy to navigate/filter from a developer's perspective and o presentable to a non-technical audience. Features ? Memory analysis for systems ranging from simple bare-metal microcontroller projects to complex projects involving multiple SoCs ? Plausibility checks of memory entries: o E.g. sections/objects outside physical memory o Labelling and handling of overlap, containment, duplicate scenarios ? Support for virtual to physical memory address translation (see "Limitations" below) ? Two types of memory region abstractions: o 1st level is user defined (e.g. SDRAM, CCM_RAM, ...) o 2nd level is fixed to: INT_RAM, INT_ROM, EXT_RAM, EXT_ROM) ? Custom section and object categorisation (based on architecture, taste, e.g. GUI, Tier1, OEM; see "Emma processing steps" below) ? Creating different summaries (see figure below): o Section_Summary: Contains the resolved* section data o Object_Summary: Contains the resolved* object data o Objects_in_Sections: Contains the merged section and object summaries * To avoid counting the same memory regions multiple times. The scenarios are labelled as overlap, containment or duplication. [objectsInSectionsResolution.png] ? Processing of GreenHills or similarly formatted map files (see also "Upcoming features" section below) ? User guidance through interactive command line ? Options for seamless integration in CI systems (--noprompt, -Werror) ? Teamscale<https://www.cqse.eu/en/teamscale/overview/> support (via non-code metrics) ? Extensive documentation and examples for a quick start Limitations * Emma requires devices to have a single address space (e.g. ARMv6M): o Some architectures have separate code and data address spaces that both start at address 0x00000000 (e.g. Intel 8051). Those architectures cannot entirely be analysed with Emma. * The stacks and heaps need to be statically defined * The virtual address spaces (VAS) need to be statically defined o Devices using memory management units (MMU) are not supported o This means that systems running complex OSs (e.g. Linux, Windows) are not supported* * In some cases this still works but you will usually loose information like the actual physical address (see here<https://github.com/bmwcarit/Emma/blob/master/doc/readme-emma.md#ghs-map-command-converter> for an example). The Emma components and the dataflow between them It is a Python-based (pip install pypiemma) tool-chain which consists of three modules: * Emma (analyser): the main executable you start with, it creates .csv data from the map files * Emma Visualiser: creates visualisations and displays consumption from the .csv data * Emma Deltas: compare/diff analysis results [workflow.png] Emma processing steps Step 1: As a common use case you would start with map files consisting of physical and/or virtual addresses. Emma will map all virtual addresses back to physical addresses using an address translation map file. Step 2: For each section and object it will be checked to which memory region it belongs. Step 3: Based on the category configuration each and every section/object is mapped to a category. After that you can start Emma Visualiser which will aggregate the .csv entries and creates automated reports and figures. [cid:image003.jpg@01D6F31A.CDCF1C10] Upcoming features ? Support for new map file formats, like GCC ? Visualisation of objects and sections for a selected memory region as .svg. -> arrives end of January 2021 Example for a simple scenario: [cid:image004.png@01D6F31A.CDCF1C10] Example for a complex scenario: [test6] I hope you found this interesting and Emma might help you facilitating your daily work. If you have further questions, feel free to reach out to us via your mailing list (emma-dev (. at) googlegroups.com). Best regards, Marcel Schmalzl and the Emma development team.