You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 20:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Fri Nov 28, 2025 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1049099280,1cyclictest12451-21apt-get10:32:422
1049099260,25cyclictest27496-21idleruntime-cro08:50:002
10496992522,2cyclictest26444-21cron11:00:003
10490992523,1cyclictest27145-21cron09:55:002
10490992522,3cyclictest0-21swapper/211:32:442
1049099250,24cyclictest0-21swapper/209:27:422
10490992420,3cyclictest0-21swapper/210:12:412
1049099241,22cyclictest1383-21basename12:20:002
1049099240,23cyclictest11970-21idleruntime-cro10:30:002
1049099240,20cyclictest4236-21apt-get12:27:422
10489992422,1cyclictest612-21gdbus12:05:001
10484992423,0cyclictest0-21swapper/010:12:410
1049699230,22cyclictest4745-21sh10:15:003
1049699230,22cyclictest16977-21apt-get11:47:423
10490992321,1cyclictest26571-21awk12:04:592
1049099231,1cyclictest32310-21grep09:00:002
1049099230,22cyclictest1707-21apt-get11:17:432
10489992321,1cyclictest8561-21dump-pmu-power12:35:001
10489992320,2cyclictest24032-21dump-pmu-power10:54:591
10489992320,2cyclictest20441-21grep07:30:011
1048999230,2cyclictest24766-21cron09:50:001
1048999230,22cyclictest11328-21apt-get11:37:431
1048499230,2cyclictest24784-21perf09:50:000
1048499230,2cyclictest10989-21apt-get12:40:000
1048499230,22cyclictest7957-21apt-get09:17:420
10496992221,1cyclictest0-21swapper/309:10:023
10496992221,0cyclictest17900-21apt-get08:32:413
10496992220,1cyclictest9577-21ls10:25:003
10496992220,1cyclictest313-21systemd-journal07:50:003
10496992220,1cyclictest22842-21apt-get09:47:443
10496992220,1cyclictest16801-21perf10:40:003
1049699221,1cyclictest18274-21apt-get07:25:013
1049699220,21cyclictest28968-21apt12:10:003
1049699220,21cyclictest1842-21apt-get12:22:423
1049699220,21cyclictest17975-21apt-get09:37:433
1049699220,21cyclictest1338-21idleruntime-cro12:20:003
1049699220,21cyclictest0-21swapper/311:17:423
1049699220,21cyclictest0-21swapper/309:17:403
1049699220,1cyclictest22903-21latency_hist07:35:003
1049699220,1cyclictest16137-21idleruntime-cro11:45:003
10490992222,0cyclictest0-21swapper/210:52:402
10490992221,1cyclictest29435-21apt-get12:12:442
10490992221,1cyclictest0-21swapper/211:52:412
10490992220,1cyclictest24187-21munin-run12:00:002
10490992220,1cyclictest17440-21munin-run08:30:002
10490992216,5cyclictest0-21swapper/209:32:422
1049099220,21cyclictest4699-21awk09:10:012
1049099220,21cyclictest0-21swapper/208:45:002
1049099220,21cyclictest0-21swapper/207:52:422
1049099220,1cyclictest22239-21apt-get08:40:002
10489992222,0cyclictest0-21swapper/112:27:411
10489992220,1cyclictest1-21systemd07:45:001
1048999220,21cyclictest28852-21grep11:05:001
1048999220,21cyclictest20296-21apt-get08:37:411
1048999220,21cyclictest13308-21apt-get07:17:431
1048999220,21cyclictest0-21swapper/111:55:031
1048999220,21cyclictest0-21swapper/109:42:401
1048999220,21cyclictest0-21swapper/107:47:411
1048999220,1cyclictest4680-21apt-get09:10:021
1048999220,1cyclictest14695-21apt-get09:30:001
10484992222,0cyclictest0-21swapper/009:02:400
10484992220,1cyclictest313-21systemd-journal09:15:010
10484992220,1cyclictest25112-21munin-run08:45:000
1048499220,21cyclictest31269-21grep11:10:000
1048499220,21cyclictest13336-21awk11:40:000
1048499220,21cyclictest10905-21apt-get07:12:410
1048499220,21cyclictest0-21swapper/011:17:430
1048499220,21cyclictest0-21swapper/011:12:410
1048499220,1cyclictest32319-21munin-run09:00:000
10496992121,0cyclictest5485-21apt-get08:07:413
10496992121,0cyclictest0-21swapper/310:27:443
10496992120,1cyclictest15722-21apt-get07:22:403
1049699210,20cyclictest0-21swapper/312:02:433
1049699210,20cyclictest0-21swapper/310:32:433
1049699210,20cyclictest0-21swapper/308:22:423
1049699210,0cyclictest30355-21apt-get08:57:413
10490992121,0cyclictest0-21swapper/211:12:412
10490992120,1cyclictest0-21swapper/207:57:412
10490992119,1cyclictest27826-21perf07:45:002
10490992119,1cyclictest16096-21cron11:45:002
10490992119,1cyclictest12247-21perf08:20:002
10490992118,3cyclictest0-21swapper/211:47:422
1049099210,20cyclictest20914-21apt-get07:32:422
1049099210,20cyclictest0-21swapper/207:27:432
1049099210,0cyclictest0-21swapper/209:47:442
10489992121,0cyclictest0-21swapper/107:52:431
10489992120,1cyclictest0-21swapper/108:04:591
10489992119,2cyclictest12316-21wc09:25:001
10489992119,1cyclictest5485-21apt-get08:07:421
10489992119,1cyclictest0-21swapper/108:45:001
1048999210,21cyclictest0-21swapper/110:02:421
1048999210,0cyclictest0-21swapper/110:27:441
1048999210,0cyclictest0-21swapper/108:52:421
1048499210,21cyclictest0-21swapper/012:22:420
1048499210,0cyclictest4236-21apt-get12:27:400
1048499210,0cyclictest0-21swapper/011:45:010
1048499210,0cyclictest0-21swapper/008:27:400
10496992020,0cyclictest9034-21apt-get12:37:443
10496992020,0cyclictest0-21swapper/310:07:443
10496992020,0cyclictest0-21swapper/309:42:403
10496992020,0cyclictest0-21swapper/308:12:433
10496992020,0cyclictest0-21swapper/307:17:433
1049699200,20cyclictest0-21swapper/311:52:423
1049699200,20cyclictest0-21swapper/310:50:013
1049699200,0cyclictest6523-21apt-get11:27:413
1049699200,0cyclictest0-21swapper/311:57:443
1049699200,0cyclictest0-21swapper/307:57:403
1049099202,17cyclictest0-21swapper/211:37:452
10490992020,0cyclictest20388-21apt-get09:42:392
10490992020,0cyclictest0-21swapper/212:07:432
10490992020,0cyclictest0-21swapper/209:17:412
10490992019,0cyclictest0-21swapper/209:22:422
10490992018,1cyclictest30214-21turbostat07:50:002
10490992018,1cyclictest0-21swapper/208:55:002
1049099200,20cyclictest0-21swapper/209:57:412
1049099200,0cyclictest0-21swapper/208:02:402
10489992020,0cyclictest381-21apt-get10:07:441
10489992020,0cyclictest0-21swapper/111:47:411
10489992020,0cyclictest0-21swapper/110:42:421
10489992020,0cyclictest0-21swapper/109:52:411
10489992020,0cyclictest0-21swapper/108:32:411
10489992020,0cyclictest0-21swapper/107:22:401
10489992019,0cyclictest0-21swapper/112:37:431
10489992018,1cyclictest0-21swapper/108:50:001
10484992020,0cyclictest0-21swapper/012:32:410
10484992020,0cyclictest0-21swapper/012:07:430
10484992020,0cyclictest0-21swapper/011:55:030
10484992020,0cyclictest0-21swapper/010:42:430
10484992020,0cyclictest0-21swapper/010:07:430
10484992020,0cyclictest0-21swapper/008:12:430
10484992020,0cyclictest0-21swapper/007:42:420
10484992019,1cyclictest0-21swapper/008:32:490
10484992019,0cyclictest15330-21apt-get09:30:000
10484992019,0cyclictest0-21swapper/009:27:410
1048499200,20cyclictest0-21swapper/012:05:000
1048499200,20cyclictest0-21swapper/011:32:440
1048499200,20cyclictest0-21swapper/010:57:440
1048499200,0cyclictest23474-21apt-get07:37:440
1048499200,0cyclictest0-21swapper/010:02:420
1048499200,0cyclictest0-21swapper/009:52:420
1048499200,0cyclictest0-21swapper/009:37:420
1048499200,0cyclictest0-21swapper/008:47:410
10496991919,0cyclictest26929-21apt-get11:02:433
10496991919,0cyclictest10282-21apt-get08:17:423
10496991919,0cyclictest0-21swapper/311:37:423
10496991919,0cyclictest0-21swapper/307:12:413
10496991918,1cyclictest31717-21apt-get11:12:413
10496991918,1cyclictest17267-21apt-get10:42:433
10496991918,1cyclictest0-21swapper/311:22:423
10496991917,1cyclictest27168-21perf09:55:003
10496991917,1cyclictest0-21swapper/310:00:013
1049699190,18cyclictest0-21swapper/312:12:423
10490991919,0cyclictest0-21swapper/212:37:442
10490991919,0cyclictest0-21swapper/207:17:422
1049099190,7cyclictest25272-21cat10:57:472
1049099190,18cyclictest7883-21apt-get08:12:422
1049099190,18cyclictest32349-21cron10:04:592
10489991919,0cyclictest0-21swapper/111:17:431
10489991919,0cyclictest0-21swapper/110:32:431
10489991919,0cyclictest0-21swapper/110:17:431
10489991919,0cyclictest0-21swapper/109:57:411
10489991919,0cyclictest0-21swapper/108:57:391
10489991919,0cyclictest0-21swapper/108:27:401
10489991919,0cyclictest0-21swapper/108:12:431
10489991919,0cyclictest0-21swapper/107:37:441
10489991919,0cyclictest0-21swapper/107:32:421
10484991919,0cyclictest30680-21apt-get07:52:410
10484991919,0cyclictest2757-21apt-get09:07:390
10484991919,0cyclictest0-21swapper/012:17:450
10484991919,0cyclictest0-21swapper/009:57:410
10484991919,0cyclictest0-21swapper/007:27:430
10484991918,1cyclictest639-21apt-get07:57:410
10496991818,0cyclictest29320-21apt-get11:07:403
10496991818,0cyclictest27958-21apt-get08:52:433
10496991818,0cyclictest0-21swapper/308:30:003
10496991818,0cyclictest0-21swapper/307:37:443
10496991816,1cyclictest7191-21awk10:19:593
10496991816,1cyclictest27506-21munin-run08:50:003
10490991818,0cyclictest0-21swapper/211:22:432
10490991818,0cyclictest0-21swapper/210:47:412
10490991818,0cyclictest0-21swapper/208:32:412
10490991818,0cyclictest0-21swapper/208:07:422
10490991817,1cyclictest13103-21apt-get08:22:432
10489991818,0cyclictest0-21swapper/111:12:411
10489991818,0cyclictest0-21swapper/110:57:431
10489991818,0cyclictest0-21swapper/110:12:421
10489991818,0cyclictest0-21swapper/109:17:411
10489991818,0cyclictest0-21swapper/109:07:391
10489991816,1cyclictest0-21swapper/110:40:001
10484991818,0cyclictest0-21swapper/010:47:410
10484991818,0cyclictest0-21swapper/008:20:020
10496991717,0cyclictest0-21swapper/309:22:413
10496991715,1cyclictest22244-21turbostat08:40:003
10496991715,1cyclictest0-21swapper/309:30:003
10490991717,0cyclictest0-21swapper/211:07:392
10490991717,0cyclictest0-21swapper/210:42:432
10490991716,0cyclictest0-21swapper/209:37:432
1049099170,17cyclictest0-21swapper/212:22:432
10489991717,0cyclictest0-21swapper/109:02:401
10489991717,0cyclictest0-21swapper/108:17:421
10489991717,0cyclictest0-21swapper/107:12:411
10489991716,0cyclictest1842-21apt-get12:22:421
10489991715,1cyclictest31379-21munin-run12:15:001
10489991714,2cyclictest8443-21ls11:30:001
10484991717,0cyclictest0-21swapper/010:27:440
10484991717,0cyclictest0-21swapper/009:42:400
10484991717,0cyclictest0-21swapper/007:47:400
10484991716,0cyclictest0-21swapper/007:17:440
1048499170,16cyclictest0-21swapper/008:17:420
10496991616,0cyclictest8922-21apt-get11:32:443
10496991616,0cyclictest0-21swapper/307:42:423
1049699160,15cyclictest2853-21seq08:00:063
10490991616,0cyclictest0-21swapper/211:02:422
10490991616,0cyclictest0-21swapper/210:07:442
10490991616,0cyclictest0-21swapper/207:12:412
1049099160,0cyclictest0-21swapper/211:27:042
1049099160,0cyclictest0-21swapper/210:21:092
10489991614,1cyclictest2617-21cat08:00:001
10489991613,2cyclictest9260-21sed10:22:521
1048999160,15cyclictest16182-21apt-config11:45:001
10484991616,0cyclictest0-21swapper/010:17:430
10484991614,1cyclictest4005-21chrt11:21:320
10484991614,1cyclictest3088-21apt-get08:02:410
10484991614,1cyclictest29889-21apt08:55:000
10496991515,0cyclictest0-21swapper/312:27:413
1049699150,1cyclictest23062-21chrt08:41:493
1049099150,1cyclictest6889-21awk10:17:522
1048999150,1cyclictest16479-21memory09:32:471
1048999150,1cyclictest0-21swapper/110:47:411
10484991515,0cyclictest14189-21apt-get11:42:420
10484991514,1cyclictest0-21swapper/011:26:520
10484991513,1cyclictest0-21swapper/008:37:440
10496991414,0cyclictest29566-21apt-get10:00:023
10496991414,0cyclictest0-21swapper/312:31:453
10496991414,0cyclictest0-21swapper/309:30:563
10496991414,0cyclictest0-21swapper/309:07:393
1049699140,14cyclictest0-21swapper/309:01:363
1049699140,0cyclictest0-21swapper/307:50:373
1049099140,14cyclictest0-21swapper/210:36:592
1049099140,14cyclictest0-21swapper/207:23:332
1049099140,0cyclictest0-21swapper/212:34:042
1049099140,0cyclictest0-21swapper/209:00:152
1049099140,0cyclictest0-21swapper/207:36:242
10489991414,0cyclictest0-21swapper/112:05:291
1048999140,14cyclictest0-21swapper/111:30:101
1048999140,14cyclictest0-21swapper/109:35:051
1048999140,14cyclictest0-21swapper/108:20:181
1048999140,0cyclictest0-21swapper/111:20:291
10484991414,0cyclictest0-21swapper/011:50:190
10484991414,0cyclictest0-21swapper/010:39:370
10484991414,0cyclictest0-21swapper/010:30:330
10484991414,0cyclictest0-21swapper/007:34:370
1048499140,14cyclictest0-21swapper/011:00:360
1048499140,14cyclictest0-21swapper/010:50:470
1048499140,14cyclictest0-21swapper/010:21:500
1048499140,14cyclictest0-21swapper/007:22:230
1048499140,0cyclictest0-21swapper/012:10:300
1048499140,0cyclictest0-21swapper/008:06:000
1049699130,0cyclictest0-21swapper/310:51:293
10490991212,0cyclictest0-21swapper/209:10:552
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional