You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-30 - 08:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot5s.osadl.org (updated Sat Aug 30, 2025 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
872965995443,7cyclictest946183-21sendmail22:40:012
872965995141,6cyclictest890758-21sendmail20:00:002
872970995043,4cyclictest878625-21munin-plugin-st19:25:003
872970995043,4cyclictest878625-21munin-plugin-st19:25:003
872956994840,4cyclictest940971-21latency_hist22:25:000
872970994612,8cyclictest947916-21basename22:45:013
872970994522,19cyclictest0-21swapper/321:35:183
872970994511,4cyclictest0-21swapper/320:27:123
872961994534,7cyclictest978336-21grep00:10:291
872961994525,11cyclictest0-21swapper/120:15:171
872965994435,4cyclictest0-21swapper/222:25:012
872956994436,4cyclictest980534-21idleruntime-cro00:20:000
87297099432,14cyclictest959335-21tail23:15:263
872961994335,4cyclictest872948-21cyclictest22:00:261
87297099422,12cyclictest923335-21tail21:30:263
872961994229,10cyclictest890004-21/usr/sbin/munin19:55:231
87297099416,4cyclictest974967-21unixbench_singl00:00:283
872970994133,5cyclictest987740-21munin-run00:39:593
872970994118,4cyclictest874584-21sh19:15:013
872970994111,6cyclictest956438-21cat23:10:003
872965994128,8cyclictest897582-21cut20:20:002
872961994131,7cyclictest904648-21apt-get20:40:131
872961994127,11cyclictest886470-21/usr/sbin/munin19:45:211
872970994019,18cyclictest0-21swapper/323:20:243
872961994033,4cyclictest975629-21apt-get00:05:171
872961994033,3cyclictest984568-21apt-get00:30:121
872961994032,4cyclictest0-21swapper/122:40:001
872961994029,7cyclictest901226-21apt-get20:30:131
872961994027,10cyclictest919659-21/usr/sbin/munin21:20:251
872961994027,10cyclictest911297-21cron20:56:011
872961994027,10cyclictest887713-21/usr/sbin/munin19:50:171
872961994027,10cyclictest879064-21cpu19:25:131
872961994025,10cyclictest895469-21/usr/sbin/munin20:10:261
872961994025,10cyclictest895469-21/usr/sbin/munin20:10:261
872961994024,13cyclictest940068-21/usr/sbin/munin22:20:211
872961994024,12cyclictest936029-21/usr/sbin/munin22:10:151
872961994024,11cyclictest934509-21/usr/sbin/munin22:05:161
872961994024,10cyclictest0-21swapper/121:50:151
872961993933,3cyclictest892701-21apt-get20:05:131
872961993933,3cyclictest892701-21apt-get20:05:121
872961993932,4cyclictest902929-21apt-get20:35:141
872961993929,7cyclictest981296-21/usr/sbin/munin00:20:191
872961993929,7cyclictest943246-21/usr/sbin/munin22:30:171
872961993929,7cyclictest907094-21/usr/sbin/munin20:45:221
872961993928,8cyclictest925861-21/usr/sbin/munin21:40:131
872961993928,7cyclictest891002-21apt-get20:00:141
872961993926,9cyclictest897923-21/usr/sbin/munin20:20:141
872961993926,8cyclictest0-21swapper/123:45:111
872961993926,10cyclictest980100-21users00:15:251
872961993926,10cyclictest948929-21/usr/sbin/munin22:45:241
872961993926,10cyclictest53-21kcompactd021:00:351
872961993925,9cyclictest974369-21/usr/sbin/munin00:00:191
872961993925,10cyclictest953301-21/usr/sbin/munin23:00:141
872961993925,10cyclictest953301-21/usr/sbin/munin23:00:131
872961993923,10cyclictest883919-21cron19:40:011
872961993922,11cyclictest946428-21apt-get22:40:111
872961993919,14cyclictest937722-21/usr/sbin/munin22:15:121
872970993830,4cyclictest980532-21kworker/u8:200:20:003
872970993815,3cyclictest0-21swapper/320:20:013
872970993813,3cyclictest966850-21sh23:40:003
872965993827,6cyclictest0-21swapper/222:35:002
872961993828,7cyclictest970900-21/usr/sbin/munin23:50:171
872961993828,7cyclictest951528-21/usr/sbin/munin22:55:291
872961993828,7cyclictest950228-21/usr/sbin/munin22:50:181
872961993828,7cyclictest914880-21/usr/sbin/munin21:05:271
872961993827,8cyclictest931577-21/usr/sbin/munin21:55:251
872961993827,7cyclictest983464-21/usr/sbin/munin00:25:251
872961993827,7cyclictest922642-21/usr/sbin/munin21:30:191
872961993827,7cyclictest881044-21/usr/sbin/munin19:30:171
872961993826,10cyclictest899867-21/usr/sbin/munin20:25:181
872961993824,11cyclictest53-21kcompactd023:11:391
872961993823,8cyclictest0-21swapper/123:20:161
872961993823,10cyclictest908340-21/usr/sbin/munin20:50:111
872956993830,4cyclictest0-21swapper/022:00:000
872956993829,4cyclictest0-21swapper/023:00:000
872970993729,5cyclictest952565-21unixbench_singl22:55:283
872965993729,5cyclictest0-21swapper/223:05:002
872965993729,5cyclictest0-21swapper/223:04:592
872961993730,4cyclictest986605-21irqstats00:35:191
872961993729,4cyclictest877147-21apt-get19:20:141
872961993729,4cyclictest877147-21apt-get19:20:141
872961993729,3cyclictest873117-21apt-get19:10:121
872961993727,7cyclictest955066-21/usr/sbin/munin23:05:151
872961993727,7cyclictest941872-21/usr/sbin/munin22:25:221
872961993727,7cyclictest927989-21/usr/sbin/munin21:45:241
872961993727,7cyclictest927989-21/usr/sbin/munin21:45:231
872961993727,7cyclictest924038-21apt-get21:35:131
872961993727,7cyclictest915644-21/usr/sbin/munin21:10:151
872961993727,7cyclictest884937-21/usr/sbin/munin19:40:271
872961993726,8cyclictest964937-21apt-get23:35:031
872961993726,8cyclictest918152-21/usr/sbin/munin21:15:271
872961993726,7cyclictest962217-21memory23:25:181
872961993726,7cyclictest958347-21/usr/sbin/munin23:15:271
872961993725,9cyclictest921841-21cron21:27:011
872961993720,14cyclictest964097-21grep23:30:241
872970993615,3cyclictest0-21swapper/321:55:003
872965993625,6cyclictest956413-21sh23:10:002
872961993627,7cyclictest973489-21cron23:58:001
872961993626,7cyclictest875764-21/usr/sbin/munin19:15:181
872961993626,7cyclictest875764-21/usr/sbin/munin19:15:171
872956993624,7cyclictest944470-21sh22:35:010
872956993623,8cyclictest964480-21grep23:30:250
872965993526,4cyclictest0-21swapper/221:10:152
872961993525,7cyclictest967938-21/usr/sbin/munin23:40:241
872970993427,4cyclictest946211-21idleruntime-cro22:40:013
872965993426,4cyclictest872948-21cyclictest21:32:262
872970993326,5cyclictest165-21systemd-journal21:17:243
872970993326,4cyclictest0-21swapper/320:20:173
872970993325,5cyclictest911638-21sh21:00:003
872965993322,7cyclictest0-21swapper/220:45:272
872956993325,3cyclictest0-21swapper/022:15:130
872956993322,7cyclictest954229-21grep23:00:250
872956993322,7cyclictest954229-21grep23:00:250
872956993322,6cyclictest0-21swapper/000:05:000
872970993225,4cyclictest977094-21kthreadd00:09:593
872970993224,5cyclictest915278-21munin-run21:10:013
872970993218,10cyclictest983408-21ntpq00:25:213
872965993224,4cyclictest966858-21latency_hist23:40:012
872965993224,4cyclictest0-21swapper/221:05:162
872965993223,6cyclictest311ktimers/223:55:132
872965993219,9cyclictest906169-21apt-key20:45:012
872956993222,6cyclictest0-21swapper/021:00:000
872956993220,9cyclictest121ktimers/019:55:140
87295699320,30cyclictest0-21swapper/019:30:140
872970993122,4cyclictest0-21swapper/322:32:343
872965993124,5cyclictest895606-21kworker/u8:2+events_unbound20:12:112
872965993124,5cyclictest895606-21kworker/u8:2+events_unbound20:12:102
872965993120,7cyclictest881386-21ps19:30:232
872956993125,3cyclictest0-21swapper/020:55:010
872956993122,6cyclictest0-21swapper/023:40:000
872956993121,6cyclictest0-21swapper/021:15:010
872970993023,4cyclictest933631-21unixbench_singl22:00:263
872970993022,5cyclictest958127-21munin-plugin-st23:14:593
872970993022,4cyclictest0-21swapper/319:30:013
872970993021,5cyclictest950891-21grep22:50:243
872970993020,6cyclictest0-21swapper/320:10:013
872970993020,6cyclictest0-21swapper/320:10:003
872970993019,8cyclictest909242-21grep20:50:243
872965993022,4cyclictest0-21swapper/222:40:262
872965993021,5cyclictest0-21swapper/200:30:152
872965993021,4cyclictest0-21swapper/221:50:202
872965993020,6cyclictest903946-21grep20:35:262
872956993023,4cyclictest0-21swapper/019:10:270
872956993020,6cyclictest0-21swapper/021:25:250
872970992922,4cyclictest0-21swapper/320:35:163
872965992922,3cyclictest0-21swapper/200:20:182
872965992921,4cyclictest0-21swapper/219:50:252
872965992921,4cyclictest0-21swapper/219:25:012
872965992921,4cyclictest0-21swapper/219:25:002
872965992920,6cyclictest971963-21sh23:55:012
872965992918,8cyclictest977099-21munin-plugin-st00:09:592
872956992921,5cyclictest886861-21unixbench_multi19:45:240
872956992920,6cyclictest0-21swapper/022:45:000
872956992920,4cyclictest0-21swapper/019:22:020
872956992920,4cyclictest0-21swapper/019:22:010
872970992818,6cyclictest890743-21cron20:00:013
872965992821,4cyclictest0-21swapper/221:25:262
872956992823,3cyclictest0-21swapper/022:25:140
872956992821,4cyclictest0-21swapper/023:40:250
872956992821,3cyclictest0-21swapper/023:55:010
872956992821,3cyclictest0-21swapper/021:05:010
872956992820,5cyclictest0-21swapper/021:05:260
872956992820,3cyclictest0-21swapper/020:10:000
872956992820,3cyclictest0-21swapper/020:10:000
872956992818,6cyclictest0-21swapper/020:20:260
872970992721,4cyclictest969643-21timerandwakeup23:45:253
872965992720,4cyclictest0-21swapper/200:10:282
872965992720,3cyclictest0-21swapper/200:15:232
872965992719,4cyclictest0-21swapper/221:40:162
872965992717,8cyclictest958092-21kworker/u8:2+events_unbound23:21:142
872956992722,2cyclictest0-21swapper/020:00:150
872956992719,4cyclictest13650irq/42-30be0000.ethernet00:30:240
872970992617,5cyclictest0-21swapper/320:35:003
872965992618,4cyclictest0-21swapper/220:00:182
872965992618,3cyclictest0-21swapper/221:20:182
872965992616,8cyclictest311ktimers/223:10:142
872965992616,7cyclictest931317-21sort21:55:202
872956992619,4cyclictest973258-21unixbench_singl23:55:250
872956992618,4cyclictest0-21swapper/000:25:170
872956992616,5cyclictest955931-21grep23:05:250
872956992614,8cyclictest215-21dbus-daemon23:27:430
872956992613,10cyclictest897583-21cat20:20:010
872970992518,4cyclictest0-21swapper/323:30:253
872970992518,4cyclictest0-21swapper/300:00:003
872970992518,3cyclictest0-21swapper/322:15:013
872970992517,4cyclictest0-21swapper/320:40:253
872965992518,4cyclictest909300-21unixbench_singl20:50:242
872965992518,3cyclictest0-21swapper/223:45:002
872965992517,5cyclictest0-21swapper/219:35:272
872965992517,4cyclictest962816-21unixbench_singl23:25:252
872965992517,4cyclictest0-21swapper/221:00:172
872965992517,4cyclictest0-21swapper/220:55:282
872965992511,11cyclictest311ktimers/220:30:142
872956992518,3cyclictest0-21swapper/021:45:010
872956992518,3cyclictest0-21swapper/021:40:010
872956992517,5cyclictest961647-21sort23:25:010
872956992517,4cyclictest0-21swapper/020:45:010
872970992418,4cyclictest966707-21kworker/u8:0+events_unbound23:44:343
872970992418,3cyclictest0-21swapper/321:40:163
872970992418,3cyclictest0-21swapper/319:15:253
872970992418,3cyclictest0-21swapper/319:15:243
872970992417,5cyclictest0-21swapper/319:40:273
872970992417,4cyclictest0-21swapper/323:50:253
872970992417,4cyclictest0-21swapper/322:45:243
872970992417,3cyclictest0-21swapper/319:30:173
872970992415,6cyclictest0-21swapper/322:25:003
872965992418,4cyclictest899172-21kworker/u8:2+events_unbound20:25:252
872965992418,4cyclictest872623-21kworker/u8:1+events_unbound19:12:012
872965992417,3cyclictest0-21swapper/222:45:252
872965992417,3cyclictest0-21swapper/219:15:032
872965992417,3cyclictest0-21swapper/219:15:032
872965992416,5cyclictest0-21swapper/222:25:252
872965992415,4cyclictest0-21swapper/200:28:492
872965992414,7cyclictest884754-21ntp_states19:40:252
872956992419,3cyclictest0-21swapper/022:00:160
872956992418,4cyclictest299-21systemd-logind19:43:410
872956992418,3cyclictest0-21swapper/019:15:030
872956992418,3cyclictest0-21swapper/019:15:030
872956992417,4cyclictest0-21swapper/023:20:010
872956992417,3cyclictest0-21swapper/023:45:170
872956992417,3cyclictest0-21swapper/019:35:210
872956992417,3cyclictest0-21swapper/019:29:070
872956992416,4cyclictest929385-21tune2fs21:50:180
872956992416,4cyclictest0-21swapper/020:10:260
872956992416,4cyclictest0-21swapper/020:10:250
872956992415,7cyclictest121ktimers/021:20:130
872956992414,6cyclictest0-21swapper/020:40:000
872956992412,8cyclictest121ktimers/000:10:170
87295699241,21cyclictest0-21swapper/000:20:030
87295699241,19cyclictest0-21swapper/022:53:080
87295699241,19cyclictest0-21swapper/000:37:480
87295699241,10cyclictest918720-21awk21:20:010
872970992317,3cyclictest0-21swapper/300:25:013
872970992316,3cyclictest0-21swapper/323:00:253
872970992316,3cyclictest0-21swapper/323:00:253
872970992315,4cyclictest939141-21cron22:20:003
872965992317,3cyclictest887353-21munin-plugin-st19:50:012
872965992317,3cyclictest0-21swapper/223:15:162
872965992316,4cyclictest918788-21latency_hist21:20:012
872965992314,7cyclictest935553-21kworker/u8:2-ext4-rsv-conversion23:00:002
872965992314,5cyclictest987733-21auditctl00:39:592
87295699238,12cyclictest121ktimers/020:25:160
872956992317,3cyclictest0-21swapper/000:05:170
872956992316,5cyclictest0-21swapper/021:35:010
872956992314,7cyclictest945385-21gzip22:35:250
872956992313,7cyclictest936516-21ntp_kernel_pll_22:10:230
872956992311,9cyclictest956793-21cut23:10:160
87295699231,18cyclictest0-21swapper/022:09:580
87295699230,19cyclictest0-21swapper/020:31:020
872970992216,3cyclictest0-21swapper/300:30:193
872970992215,4cyclictest391ktimers/321:25:353
872970992214,5cyclictest0-21swapper/319:50:173
87296599222,14cyclictest899305-21cut20:25:002
872965992216,3cyclictest0-21swapper/222:05:242
872965992216,3cyclictest0-21swapper/222:00:202
872965992215,5cyclictest930507-21kworker/u8:1+events_unbound22:12:332
872965992214,5cyclictest311ktimers/219:25:052
872965992214,4cyclictest0-21swapper/222:50:252
872956992217,3cyclictest0-21swapper/022:45:180
872956992214,5cyclictest888433-21ls19:50:250
87295699221,19cyclictest0-21swapper/020:45:220
872970992115,3cyclictest0-21swapper/321:55:233
872970992115,3cyclictest0-21swapper/320:45:193
872970992114,4cyclictest894949-21ntpq20:10:213
872970992114,4cyclictest894949-21ntpq20:10:203
872970992113,5cyclictest886869-21unixbench_multi19:45:233
872970992113,5cyclictest0-21swapper/321:00:253
872970992113,5cyclictest0-21swapper/319:35:183
872970992112,6cyclictest962800-21grep23:25:253
87296599219,8cyclictest964957-21basename23:35:012
872965992115,3cyclictest0-21swapper/221:45:302
872965992115,3cyclictest0-21swapper/221:45:302
872965992114,5cyclictest311ktimers/222:15:112
872965992113,5cyclictest892787-21awk20:05:152
872965992113,5cyclictest892787-21awk20:05:152
872970992015,3cyclictest890999-21/usr/sbin/munin20:00:193
872970992014,3cyclictest0-21swapper/322:05:253
872970992012,5cyclictest0-21swapper/321:45:203
872970992012,5cyclictest0-21swapper/321:45:193
872970992010,7cyclictest977772-21/usr/sbin/munin00:10:233
87296599207,7cyclictest968785-21date23:45:012
872970991912,5cyclictest391ktimers/321:10:133
872970991911,5cyclictest391ktimers/322:25:133
872970991911,5cyclictest0-21swapper/321:20:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional