You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-20 - 09:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Mon Apr 20, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2712499260,1cyclictest24807-21sh21:45:040
2712499260,1cyclictest0-21swapper/023:07:300
2713599250,24cyclictest23491-21ssh22:46:173
2713599250,24cyclictest21001-21ssh22:12:563
2713599250,24cyclictest10917-21ssh22:03:023
27133992522,2cyclictest1-21systemd19:40:002
2713399250,24cyclictest16350-21ssh23:44:372
27124992523,2cyclictest0-21swapper/000:01:520
27124992521,3cyclictest20780-21diskmemload21:55:030
27124992521,3cyclictest0-21swapper/021:37:520
27124992521,3cyclictest0-21swapper/021:37:520
27124992521,3cyclictest0-21swapper/000:17:520
2712499250,24cyclictest20780-21diskmemload22:21:540
2712499250,24cyclictest0-21swapper/023:14:560
2712499250,21cyclictest30419-21ssh22:54:190
2712499250,21cyclictest0-21swapper/022:26:520
2712499250,0cyclictest0-21swapper/022:11:260
27135992423,1cyclictest15454-21ssh22:07:113
2713599240,24cyclictest0-21swapper/323:39:313
2713599240,23cyclictest27518-21ssh00:25:463
2713599240,23cyclictest20780-21diskmemload00:04:083
2713599240,23cyclictest15619-21grep21:00:013
2713399240,23cyclictest9387-21sh23:36:252
2713399240,23cyclictest25204-21ssh22:49:022
27128992423,1cyclictest2229-21ssh22:58:011
27128992423,1cyclictest0-21swapper/122:46:051
27128992422,1cyclictest0-21swapper/122:13:131
2712899240,24cyclictest0-21swapper/123:53:041
2712899240,24cyclictest0-21swapper/122:08:461
2712899240,23cyclictest3060-21ssh22:26:391
2712899240,0cyclictest20780-21diskmemload23:26:451
27124992422,1cyclictest0-21swapper/023:24:330
27124992422,1cyclictest0-21swapper/000:36:130
27124992421,3cyclictest0-21swapper/000:31:280
27124992421,2cyclictest0-21swapper/000:26:430
27124992420,3cyclictest0-21swapper/021:27:580
27124992418,5cyclictest0-21swapper/019:30:000
2712499240,23cyclictest9553-21cron00:10:010
2712499240,23cyclictest20780-21diskmemload23:34:210
2712499240,22cyclictest32310-21/usr/sbin/munin21:20:090
2712499240,20cyclictest0-21swapper/022:57:170
2712499240,0cyclictest10218-21ssh23:37:440
27135992323,0cyclictest20780-21diskmemload00:38:523
27135992323,0cyclictest0-21swapper/323:58:033
27135992323,0cyclictest0-21swapper/322:57:013
27135992323,0cyclictest0-21swapper/322:43:113
27135992323,0cyclictest0-21swapper/322:19:073
27135992323,0cyclictest0-21swapper/321:55:583
27135992323,0cyclictest0-21swapper/321:39:123
27135992323,0cyclictest0-21swapper/321:39:123
27135992323,0cyclictest0-21swapper/321:16:113
27135992323,0cyclictest0-21swapper/300:17:453
27135992323,0cyclictest0-21swapper/300:12:063
27135992321,1cyclictest0-21swapper/322:28:403
27135992320,2cyclictest30811-21munin-plugin-st22:55:003
2713599230,23cyclictest0-21swapper/323:20:053
2713599230,22cyclictest8121-21ssh23:04:233
2713599230,22cyclictest7769-21sh00:07:133
2713599230,22cyclictest31315-21ssh23:25:593
2713599230,22cyclictest24092-21ssh23:50:473
2713599230,22cyclictest13755-21ssh22:36:573
2713599230,0cyclictest20780-21diskmemload23:12:443
2713599230,0cyclictest20780-21diskmemload21:34:533
2713599230,0cyclictest0-21swapper/322:21:353
2713599230,0cyclictest0-21swapper/321:23:273
2713599230,0cyclictest0-21swapper/300:30:583
27133992323,0cyclictest0-21swapper/223:22:072
27133992323,0cyclictest0-21swapper/221:10:072
27133992322,1cyclictest11584-21ssh23:06:462
27133992321,1cyclictest10811-21grep20:50:012
27133992321,1cyclictest10516-21ssh22:02:152
27133992321,1cyclictest0-21swapper/222:52:552
27133992321,1cyclictest0-21swapper/200:40:002
27133992320,2cyclictest10017-21ssh22:34:492
2713399230,23cyclictest0-21swapper/222:24:342
2713399230,23cyclictest0-21swapper/200:25:192
2713399230,23cyclictest0-21swapper/200:17:212
2713399230,22cyclictest2746-21sh22:58:572
2713399230,22cyclictest26350-21munin-run20:15:012
2713399230,22cyclictest0-21swapper/223:10:312
2713399230,22cyclictest0-21swapper/222:35:382
2713399230,0cyclictest0-21swapper/223:26:512
2713399230,0cyclictest0-21swapper/221:20:112
2713399230,0cyclictest0-21swapper/219:40:022
2713399230,0cyclictest0-21swapper/200:20:542
27128992323,0cyclictest0-21swapper/123:33:101
27128992323,0cyclictest0-21swapper/122:54:051
27128992323,0cyclictest0-21swapper/122:35:471
27128992323,0cyclictest0-21swapper/122:20:191
27128992321,1cyclictest5617-21ssh00:36:201
27128992321,1cyclictest18159-21sh21:39:001
27128992321,1cyclictest18159-21sh21:39:001
27128992321,1cyclictest0-21swapper/122:40:101
27128992321,1cyclictest0-21swapper/122:16:091
27128992321,1cyclictest0-21swapper/121:15:091
27128992320,2cyclictest32508-21pmu-power21:20:111
27128992320,2cyclictest11070-21ssh23:05:501
2712899230,23cyclictest20780-21diskmemload23:46:351
2712899230,23cyclictest0-21swapper/121:45:131
2712899230,22cyclictest20780-21diskmemload23:35:111
2712899230,22cyclictest13325-21sh00:13:051
2712899230,22cyclictest12591-21grep23:40:061
2712899230,0cyclictest20780-21diskmemload00:05:111
2712899230,0cyclictest0-21swapper/123:55:181
2712899230,0cyclictest0-21swapper/123:10:391
2712899230,0cyclictest0-21swapper/121:56:201
2712899230,0cyclictest0-21swapper/121:10:131
27124992323,0cyclictest0-21swapper/022:46:190
27124992323,0cyclictest0-21swapper/022:35:220
27124992323,0cyclictest0-21swapper/000:14:530
27124992321,2cyclictest0-21swapper/023:53:020
27124992321,1cyclictest8982-21apt-get21:30:030
27124992321,1cyclictest20780-21diskmemload23:15:580
27124992321,1cyclictest0-21swapper/023:56:560
27124992321,1cyclictest0-21swapper/023:45:310
27124992321,1cyclictest0-21swapper/022:40:210
27124992320,3cyclictest0-21swapper/022:06:490
27124992320,2cyclictest20780-21diskmemload00:21:110
27124992317,5cyclictest0-21swapper/021:10:030
2712499230,23cyclictest20780-21diskmemload22:30:230
2712499230,22cyclictest19391-21cstates21:40:050
2712499230,22cyclictest0-21swapper/022:15:220
2712499230,22cyclictest0-21swapper/022:04:380
2712499230,0cyclictest20780-21diskmemload23:28:420
27135992222,0cyclictest0-21swapper/323:30:113
27135992221,1cyclictest17883-21ssh23:45:083
27135992221,0cyclictest0-21swapper/320:25:043
27135992220,1cyclictest23910-21apt-get21:45:023
27135992220,1cyclictest0-21swapper/321:50:133
27135992220,1cyclictest0-21swapper/321:10:073
27135992220,1cyclictest0-21swapper/300:20:083
2713599220,22cyclictest0-21swapper/323:09:303
2713599220,22cyclictest0-21swapper/321:43:463
2713599220,22cyclictest0-21swapper/321:25:323
2713599220,21cyclictest20780-21diskmemload23:42:083
2713599220,21cyclictest20780-21diskmemload22:30:583
2713599220,21cyclictest1649-21cron19:35:003
2713599220,21cyclictest0-21swapper/320:00:033
2713599220,1cyclictest26601-21apt-get20:15:053
2713599220,1cyclictest19145-21dump-pmu-power20:00:013
2713599220,1cyclictest19145-21dump-pmu-power20:00:013
2713599220,1cyclictest11061-21apt-get20:50:043
2713599220,1cyclictest0-21swapper/319:30:003
2713599220,0cyclictest0-21swapper/323:16:593
27133992222,0cyclictest20780-21diskmemload21:45:472
27133992222,0cyclictest20780-21diskmemload21:15:552
27133992222,0cyclictest0-21swapper/200:04:362
27133992221,1cyclictest13951-21apt-get19:50:022
27133992220,1cyclictest8982-21apt-get21:30:042
27133992220,1cyclictest0-21swapper/223:56:412
27133992220,1cyclictest0-21swapper/200:10:052
2713399220,21cyclictest20780-21diskmemload21:50:052
2713399220,21cyclictest18839-21ssh23:45:202
2713399220,21cyclictest1528-21ssh00:32:542
2713399220,21cyclictest0-21swapper/223:50:222
2713399220,21cyclictest0-21swapper/223:30:042
2713399220,21cyclictest0-21swapper/222:05:112
2713399220,21cyclictest0-21swapper/221:55:092
2713399220,21cyclictest0-21swapper/221:40:052
2713399220,21cyclictest0-21swapper/221:05:042
2713399220,0cyclictest20780-21diskmemload22:44:192
2713399220,0cyclictest0-21swapper/223:03:292
2713399220,0cyclictest0-21swapper/219:15:012
27128992222,0cyclictest0-21swapper/121:30:491
27128992222,0cyclictest0-21swapper/121:26:151
27128992222,0cyclictest0-21swapper/100:15:051
27128992222,0cyclictest0-21swapper/100:01:111
27128992220,1cyclictest5951-21perf20:40:001
27128992220,1cyclictest25768-21diskstats00:25:061
2712899220,21cyclictest3561-21perf20:35:001
2712899220,21cyclictest23984-21ssh23:20:041
2712899220,21cyclictest20780-21diskmemload23:15:051
2712899220,21cyclictest1121-21apt20:30:001
2712899220,21cyclictest0-21swapper/123:00:131
2712899220,21cyclictest0-21swapper/120:00:031
2712899220,1cyclictest0-21swapper/122:31:341
2712899220,1cyclictest0-21swapper/122:04:011
2712899220,1cyclictest0-21swapper/100:30:471
2712899220,1cyclictest0-21swapper/100:21:531
2712899220,0cyclictest0-21swapper/121:43:121
27124992222,0cyclictest0-21swapper/019:10:020
2712499220,21cyclictest606-21dbus-daemon21:00:010
2712499220,21cyclictest3556-21ls20:35:000
2712499220,21cyclictest20813-21munin-run21:10:000
2712499220,21cyclictest19164-21munin-plugin-st20:00:010
2712499220,21cyclictest19164-21munin-plugin-st20:00:010
2712499220,21cyclictest0-21swapper/021:15:490
2712499220,1cyclictest8430-21ssh23:04:580
2712499220,1cyclictest5951-21perf20:40:000
2712499220,0cyclictest0-21swapper/023:40:240
27135992120,0cyclictest0-21swapper/320:30:043
27133992121,0cyclictest0-21swapper/223:15:072
27133992120,1cyclictest0-21swapper/222:15:152
27133992120,0cyclictest3919-21apt-get21:25:042
2713399210,21cyclictest0-21swapper/219:45:032
2713399210,20cyclictest2156-21apt-get19:25:032
27128992120,1cyclictest0-21swapper/121:50:381
27128992119,1cyclictest28745-21perf20:20:001
27128992119,1cyclictest24206-21apt-get20:10:041
2712899210,0cyclictest0-21swapper/121:00:041
2712899210,0cyclictest0-21swapper/119:25:031
2712899210,0cyclictest0-21swapper/119:20:041
27124992121,0cyclictest0-21swapper/019:40:020
27124992120,0cyclictest0-21swapper/020:25:040
2712499210,20cyclictest0-21swapper/019:50:050
27135992020,0cyclictest9366-21apt-get19:40:023
27135992019,0cyclictest32173-21apt-get19:20:043
2713599200,20cyclictest0-21swapper/319:35:023
2713599200,0cyclictest29026-21apt-get20:20:043
2713599200,0cyclictest23953-21perf20:10:013
2713599200,0cyclictest0-21swapper/319:20:013
27133992020,0cyclictest0-21swapper/221:00:022
27133992020,0cyclictest0-21swapper/220:00:032
27133992019,0cyclictest0-21swapper/220:15:052
2713399200,0cyclictest0-21swapper/219:15:032
2712899200,20cyclictest0-21swapper/119:50:051
27124992020,0cyclictest0-21swapper/019:35:020
27124992020,0cyclictest0-21swapper/019:30:020
27124992019,1cyclictest0-21swapper/020:05:030
27135991918,1cyclictest6219-21apt-get20:40:033
27135991917,1cyclictest16746-21idleruntime-cro19:55:013
27135991917,1cyclictest15588-21apt-get21:00:023
27133991919,0cyclictest16999-21apt-get19:55:042
27133991919,0cyclictest16999-21apt-get19:55:042
27133991918,1cyclictest31420-21apt-get20:25:042
27133991917,1cyclictest0-21swapper/221:00:012
27128991919,0cyclictest0-21swapper/119:10:021
27124991918,1cyclictest16266-21apt-get21:00:030
2712499190,18cyclictest13175-21cron20:55:010
27135991818,0cyclictest27360-21apt-get19:10:033
27135991818,0cyclictest0-21swapper/320:10:043
27135991818,0cyclictest0-21swapper/319:45:033
27133991818,0cyclictest0-21swapper/220:20:042
27128991817,0cyclictest11814-21apt-get19:45:031
27124991816,1cyclictest28722-21cron20:20:000
2713399170,16cyclictest0-21swapper/219:30:022
2713399170,0cyclictest0-21swapper/220:50:042
27128991717,0cyclictest0-21swapper/120:50:041
27128991717,0cyclictest0-21swapper/119:30:021
27128991717,0cyclictest0-21swapper/119:15:031
2712899170,0cyclictest0-21swapper/121:05:041
27124991717,0cyclictest0-21swapper/020:40:030
27124991717,0cyclictest0-21swapper/020:10:040
2712499170,1cyclictest0-21swapper/020:00:030
2712899160,14cyclictest0-21swapper/120:20:061
27124991616,0cyclictest0-21swapper/019:20:040
2712499160,0cyclictest0-21swapper/019:16:510
2713599151,10cyclictest0-21swapper/320:35:033
2713599150,1cyclictest0-21swapper/321:10:003
27133991515,0cyclictest0-21swapper/220:35:032
27133991515,0cyclictest0-21swapper/219:20:042
2712899154,9cyclictest7284-21/usr/sbin/munin20:40:091
27128991515,0cyclictest0-21swapper/120:45:461
2712499152,3cyclictest31175-21perf20:25:000
2712499150,15cyclictest0-21swapper/020:49:180
2712499150,11cyclictest783-21snmpd19:48:080
2713599140,14cyclictest0-21swapper/320:45:373
2713399140,14cyclictest0-21swapper/220:44:582
2713399140,14cyclictest0-21swapper/220:32:322
27128991414,0cyclictest0-21swapper/120:05:051
2712899140,0cyclictest0-21swapper/120:55:111
2712899140,0cyclictest0-21swapper/119:35:251
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional