You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-30 - 03:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Tue Jun 30, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20268992822,5cyclictest23548-21ssh22:43:563
2026899280,27cyclictest0-21swapper/322:31:273
2026899280,27cyclictest0-21swapper/321:30:573
20266992823,4cyclictest26648-21apt-get21:20:582
2026699280,21cyclictest0-21swapper/219:20:582
2026699270,26cyclictest0-21swapper/221:30:082
2026699270,1cyclictest0-21swapper/200:21:032
20268992623,3cyclictest0-21swapper/322:02:173
20268992620,5cyclictest14313-21diskmemload00:07:043
20268992620,5cyclictest0-21swapper/300:15:013
2026899260,2cyclictest17213-21grep23:06:083
2026899260,25cyclictest0-21swapper/322:16:383
20266992622,3cyclictest0-21swapper/200:33:392
20266992621,1cyclictest15888-21diskstats23:06:002
2026699260,26cyclictest0-21swapper/223:29:032
2026699260,25cyclictest0-21swapper/223:56:172
2026699260,22cyclictest23418-21sh23:11:502
2026199260,25cyclictest5433-21ssh00:22:231
20268992520,4cyclictest0-21swapper/322:20:563
2026899250,24cyclictest5352-21ssh23:25:473
2026899250,24cyclictest14313-21diskmemload00:25:543
2026899250,24cyclictest0-21swapper/323:24:123
2026899250,24cyclictest0-21swapper/322:59:413
2026899250,24cyclictest0-21swapper/322:13:273
2026899250,24cyclictest0-21swapper/321:57:543
2026899250,21cyclictest29153-21sh22:48:503
20266992523,2cyclictest0-21swapper/222:24:212
20266992521,4cyclictest0-21swapper/222:00:462
2026699250,25cyclictest0-21swapper/222:05:572
2026699250,24cyclictest30143-21ssh23:46:442
2026699250,24cyclictest2994-21ssh23:51:152
2026699250,21cyclictest0-21swapper/222:37:522
20261992522,2cyclictest21331-21sed00:36:081
2026199250,24cyclictest0-21swapper/121:47:131
20268992422,1cyclictest0-21swapper/323:55:003
20268992420,3cyclictest0-21swapper/323:34:413
20268992419,4cyclictest0-21swapper/323:56:373
20268992419,4cyclictest0-21swapper/300:36:303
2026899240,24cyclictest0-21swapper/321:42:023
2026899240,23cyclictest0-21swapper/321:10:203
2026899240,1cyclictest0-21swapper/300:20:403
2026899240,0cyclictest0-21swapper/323:17:413
2026899240,0cyclictest0-21swapper/323:02:503
2026899240,0cyclictest0-21swapper/322:52:013
2026699240,24cyclictest0-21swapper/221:28:262
2026699240,23cyclictest0-21swapper/222:57:152
2026699240,23cyclictest0-21swapper/221:10:072
2026699240,23cyclictest0-21swapper/200:18:282
2026699240,20cyclictest0-21swapper/222:25:492
20261992424,0cyclictest0-21swapper/122:56:481
20261992422,1cyclictest0-21swapper/122:51:021
2026199240,24cyclictest0-21swapper/123:25:521
2026199240,1cyclictest30424-21ssh23:18:481
20258992423,1cyclictest62550irq/127-eno122:50:130
2025899240,24cyclictest0-21swapper/023:30:520
2025899240,24cyclictest0-21swapper/022:00:180
2025899240,24cyclictest0-21swapper/021:44:360
2025899240,1cyclictest0-21swapper/021:49:390
2025899240,0cyclictest0-21swapper/023:58:000
2025899240,0cyclictest0-21swapper/022:56:090
20268992323,0cyclictest0-21swapper/323:37:033
20268992321,2cyclictest0-21swapper/322:28:593
20268992320,3cyclictest0-21swapper/321:53:003
20268992320,3cyclictest0-21swapper/300:10:133
2026899230,2cyclictest11946-21latency_hist21:05:013
2026899230,23cyclictest0-21swapper/322:06:023
2026899230,23cyclictest0-21swapper/321:36:433
2026899230,22cyclictest17922-21sh22:39:293
2026899230,22cyclictest14700-21perf20:05:003
2026899230,22cyclictest14313-21diskmemload21:18:293
2026899230,0cyclictest0-21swapper/321:29:033
2026899230,0cyclictest0-21swapper/300:00:063
20266992323,0cyclictest0-21swapper/200:25:422
20266992323,0cyclictest0-21swapper/200:11:112
20266992323,0cyclictest0-21swapper/200:08:412
20266992321,2cyclictest14313-21diskmemload21:45:492
20266992321,1cyclictest0-21swapper/223:20:012
20266992321,0cyclictest0-21swapper/221:56:462
20266992320,2cyclictest313-21systemd-journal20:05:002
20266992318,4cyclictest0-21swapper/221:36:492
2026699230,2cyclictest313-21systemd-journal19:15:002
2026699230,23cyclictest0-21swapper/223:42:562
2026699230,23cyclictest0-21swapper/221:50:392
2026699230,22cyclictest9592-21ssh23:00:362
2026699230,22cyclictest31657-21grep22:51:022
2026699230,22cyclictest10431-21sh22:31:412
2026699230,20cyclictest14313-21diskmemload22:45:252
2026699230,0cyclictest0-21swapper/223:39:042
2026699230,0cyclictest0-21swapper/222:10:282
2026699230,0cyclictest0-21swapper/221:15:572
2026699230,0cyclictest0-21swapper/200:02:192
20261992323,0cyclictest0-21swapper/122:01:001
20261992323,0cyclictest0-21swapper/121:10:471
20261992323,0cyclictest0-21swapper/100:15:421
20261992322,1cyclictest18002-21apt-get00:06:031
20261992322,1cyclictest14313-21diskmemload21:31:161
20261992321,1cyclictest606-21dbus-daemon23:10:001
20261992321,1cyclictest491-21ssh21:55:241
20261992321,1cyclictest19697-21apt-get22:40:591
20261992321,1cyclictest0-21swapper/120:15:011
20261992320,2cyclictest4228-21grep22:26:091
20261992320,2cyclictest20319-21sh21:15:101
20261992320,2cyclictest19751-21ssh22:11:301
20261992320,2cyclictest10557-21rm23:58:581
2026199231,1cyclictest17098-21munin-plugin-st20:10:001
2026199230,23cyclictest0-21swapper/123:53:011
2026199230,23cyclictest0-21swapper/123:40:161
2026199230,23cyclictest0-21swapper/123:37:321
2026199230,23cyclictest0-21swapper/123:21:131
2026199230,23cyclictest0-21swapper/122:19:451
2026199230,23cyclictest0-21swapper/121:44:571
2026199230,23cyclictest0-21swapper/121:38:291
2026199230,22cyclictest27621-21diskmemload23:45:361
2026199230,22cyclictest22780-21sh00:10:041
2026199230,22cyclictest14815-21ssh00:01:531
2026199230,22cyclictest14313-21diskmemload00:31:571
2026199230,22cyclictest11347-21ssh22:32:581
2026199230,22cyclictest0-21swapper/123:30:571
2026199230,0cyclictest14313-21diskmemload22:22:451
2026199230,0cyclictest0-21swapper/122:36:251
2026199230,0cyclictest0-21swapper/122:06:481
2026199230,0cyclictest0-21swapper/121:26:041
20258992323,0cyclictest0-21swapper/023:25:400
20258992323,0cyclictest0-21swapper/022:41:540
20258992323,0cyclictest0-21swapper/022:24:180
20258992323,0cyclictest0-21swapper/021:10:100
20258992323,0cyclictest0-21swapper/000:25:010
20258992321,1cyclictest30072-21ssh23:46:390
20258992321,1cyclictest0-21swapper/021:51:350
20258992320,2cyclictest981-21ssh23:21:030
2025899230,2cyclictest26654-21ssh23:15:580
2025899230,2cyclictest25065-21sh00:11:090
2025899230,2cyclictest17936-21ssh23:06:570
2025899230,23cyclictest0-21swapper/022:33:320
2025899230,23cyclictest0-21swapper/021:16:330
2025899230,22cyclictest9502-21ssh23:00:290
2025899230,22cyclictest31967-21awk20:40:000
2025899230,22cyclictest27828-21ssh22:47:020
2025899230,22cyclictest23962-21ssh23:12:370
2025899230,22cyclictest14313-21diskmemload23:50:160
2025899230,22cyclictest14313-21diskmemload22:17:530
2025899230,22cyclictest14313-21diskmemload21:32:100
2025899230,22cyclictest0-21swapper/021:55:570
2025899230,1cyclictest31386-21ssh00:16:140
2025899230,1cyclictest0-21swapper/022:05:570
2025899230,1cyclictest0-21swapper/019:40:580
2025899230,0cyclictest14313-21diskmemload21:28:360
2025899230,0cyclictest14313-21diskmemload00:38:590
2025899230,0cyclictest0-21swapper/023:41:510
2025899230,0cyclictest0-21swapper/022:35:020
2025899230,0cyclictest0-21swapper/021:37:140
2025899230,0cyclictest0-21swapper/000:30:360
20268992220,2cyclictest0-21swapper/323:45:343
20268992220,1cyclictest22284-21turbostat.cron20:20:013
20268992220,1cyclictest0-21swapper/323:41:043
20268992220,1cyclictest0-21swapper/321:20:033
20268992217,4cyclictest0-21swapper/321:45:483
2026899220,21cyclictest13531-21rm00:30:233
2026899220,1cyclictest14313-21diskmemload23:11:073
20266992222,0cyclictest0-21swapper/222:41:012
20266992222,0cyclictest0-21swapper/221:40:032
20266992220,1cyclictest0-21swapper/220:10:022
2026699220,22cyclictest0-21swapper/200:36:122
2026699220,21cyclictest4668-21apt-get20:50:592
2026699220,21cyclictest0-21swapper/222:17:102
2026699220,21cyclictest0-21swapper/219:50:572
20261992220,1cyclictest7712-21ssh00:25:121
2026199220,21cyclictest20890-21rm23:10:511
2026199220,21cyclictest0-21swapper/123:04:201
2026199220,21cyclictest0-21swapper/121:20:031
20258992222,0cyclictest32619-21apt-get19:35:580
20258992222,0cyclictest14313-21diskmemload00:21:150
2025899220,22cyclictest19601-21timerandwakeup00:06:130
2025899220,22cyclictest0-21swapper/023:35:520
2025899220,1cyclictest26333-21ssh21:20:360
2025899220,1cyclictest0-21swapper/022:25:170
2025899220,1cyclictest0-21swapper/000:01:020
20268992120,0cyclictest30226-21apt-get19:30:573
20266992120,1cyclictest9875-21apt-get21:00:582
20266992120,0cyclictest0-21swapper/219:30:582
20266992119,1cyclictest27472-21apt-get20:30:572
2026699210,21cyclictest2489-1kworker/u9:123:20:032
2026699210,0cyclictest0-21swapper/220:21:002
2026699210,0cyclictest0-21swapper/219:45:572
20261992121,0cyclictest0-21swapper/119:15:581
2026199210,21cyclictest0-21swapper/120:55:021
2026199210,21cyclictest0-21swapper/120:20:011
2026199210,0cyclictest0-21swapper/120:00:591
20258992121,0cyclictest0-21swapper/021:05:570
20258992121,0cyclictest0-21swapper/020:51:000
2025899210,21cyclictest0-21swapper/019:55:570
2025899210,0cyclictest0-21swapper/019:30:580
20268992020,0cyclictest0-21swapper/321:05:573
20268992020,0cyclictest0-21swapper/320:35:573
20268992020,0cyclictest0-21swapper/319:20:583
20268992018,1cyclictest606-21dbus-daemon20:00:003
20268992016,4cyclictest0-21swapper/319:45:563
20268992015,5cyclictest0-21swapper/319:40:583
20266992020,0cyclictest0-21swapper/219:40:582
20266992020,0cyclictest0-21swapper/219:35:582
20266992019,0cyclictest9808-21apt-get19:55:572
20261992020,0cyclictest0-21swapper/119:50:571
20261992020,0cyclictest0-21swapper/119:10:581
20261992019,0cyclictest0-21swapper/119:55:571
2026199200,0cyclictest0-21swapper/121:05:581
20258992020,0cyclictest0-21swapper/019:20:580
20268991918,0cyclictest0-21swapper/320:05:573
20266991919,0cyclictest0-21swapper/220:15:552
20266991919,0cyclictest0-21swapper/220:05:572
20266991918,1cyclictest0-21swapper/220:25:592
20261991919,0cyclictest5018-21apt-get19:45:561
20261991919,0cyclictest0-21swapper/120:35:571
20261991918,1cyclictest0-21swapper/119:30:581
20261991917,1cyclictest29554-21munin-run20:35:001
20258991919,0cyclictest32287-21apt-get20:40:580
20258991919,0cyclictest0-21swapper/021:00:580
20258991919,0cyclictest0-21swapper/019:25:580
20258991917,1cyclictest22303-21perf20:20:010
20268991818,0cyclictest0-21swapper/320:30:573
20268991817,1cyclictest2268-21apt-get20:45:563
20268991816,1cyclictest0-21swapper/320:21:003
20268991815,2cyclictest9512-21cron21:00:003
20266991818,0cyclictest0-21swapper/220:55:022
20266991816,1cyclictest619-21polkitd19:30:012
20261991818,0cyclictest0-21swapper/120:45:561
20261991818,0cyclictest0-21swapper/120:40:581
20268991717,0cyclictest0-21swapper/319:10:583
20268991714,2cyclictest29938-21latency_hist19:30:013
20261991717,0cyclictest0-21swapper/120:25:591
20261991717,0cyclictest0-21swapper/119:35:581
20258991717,0cyclictest20567-21apt-get19:10:580
20258991717,0cyclictest0-21swapper/020:00:580
20258991717,0cyclictest0-21swapper/019:50:570
2025899170,1cyclictest29516-21cron20:35:000
20268991614,1cyclictest0-21swapper/319:40:013
2026899160,16cyclictest0-21swapper/319:54:173
20266991614,1cyclictest606-21dbus-daemon20:45:002
20266991614,1cyclictest0-21swapper/220:46:042
20266991614,1cyclictest0-21swapper/220:36:122
20261991616,0cyclictest0-21swapper/121:00:581
20258991616,0cyclictest15014-21apt-get20:05:570
20258991616,0cyclictest0-21swapper/019:45:570
20258991615,1cyclictest0-21swapper/019:15:580
2025899160,15cyclictest7425-21/usr/sbin/munin20:56:010
20268991514,1cyclictest0-21swapper/320:10:573
20261991515,0cyclictest0-21swapper/119:40:581
2025899157,7cyclictest62550irq/127-eno120:21:000
2025899150,14cyclictest0-21swapper/020:11:050
20268991414,0cyclictest0-21swapper/320:51:503
20268991412,1cyclictest23965-21cut19:16:033
20268991411,0cyclictest0-21swapper/320:25:013
2026899140,0cyclictest0-21swapper/320:41:033
2026699140,14cyclictest0-21swapper/221:05:472
2026699140,0cyclictest22962-21apt-get19:15:582
20261991414,0cyclictest0-21swapper/119:20:481
2026199140,0cyclictest0-21swapper/120:53:161
20258991414,0cyclictest0-21swapper/020:27:070
2025899140,0cyclictest0-21swapper/020:45:580
20261991212,0cyclictest0-21swapper/120:21:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional