You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 17:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun Nov 30, 2025 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3737992624,1cyclictest593-21dbus-daemon11:15:001
3739992523,1cyclictest31344-21apt11:25:003
373799250,24cyclictest19302-21cron11:00:001
3739992422,1cyclictest9280-21perf10:40:003
373699240,23cyclictest29598-21idleruntime-cro09:10:000
3739992323,0cyclictest0-21swapper/311:37:533
3739992320,2cyclictest12687-21apt-get09:40:023
373999230,22cyclictest593-21dbus-daemon12:40:003
3738992320,2cyclictest17249-21basename09:50:002
3737992321,1cyclictest0-21swapper/111:35:001
373799231,21cyclictest19833-21apt-get11:02:551
373799230,2cyclictest24464-21cron10:05:011
3736992323,0cyclictest0-21swapper/008:57:540
3736992320,2cyclictest8967-21latency_hist11:45:010
3739992220,1cyclictest21037-21awk12:10:003
3739992220,1cyclictest0-21swapper/309:12:533
373999220,21cyclictest9159-21apt-get07:20:023
373999220,21cyclictest3573-21apt-get08:17:523
373999220,21cyclictest14290-21apt-get11:57:543
373999220,1cyclictest2220-21apt-get11:32:533
3738992220,1cyclictest1999-21munin-run09:20:012
373899220,21cyclictest0-21swapper/207:22:512
373899220,1cyclictest4407-21latency_hist09:25:002
3737992221,1cyclictest7353-21apt-get10:37:551
3737992221,0cyclictest19181-21apt-get07:42:521
3737992220,1cyclictest15015-21apt-get10:52:541
3737992220,1cyclictest0-21swapper/110:20:001
373799220,21cyclictest562-21idleruntime-cro12:34:591
373799220,21cyclictest26862-21turbostat.cron10:10:011
373799220,21cyclictest24147-21grep11:10:001
373799220,21cyclictest20133-21apt-get09:57:531
373799220,21cyclictest14264-21apt-get07:32:521
373799220,21cyclictest0-21swapper/112:27:541
373799220,21cyclictest0-21swapper/107:47:541
373799220,1cyclictest11754-21munin-run10:45:001
373799220,1cyclictest0-21swapper/109:07:531
373699220,21cyclictest30138-21apt-get10:17:530
373699220,21cyclictest28958-21grep11:20:000
373699220,21cyclictest0-21swapper/012:37:550
373699220,21cyclictest0-21swapper/012:32:550
373699220,1cyclictest9293-21awk10:40:000
3739992121,0cyclictest0-21swapper/310:52:553
3739992121,0cyclictest0-21swapper/307:57:523
3739992120,1cyclictest1165-21apt-get08:12:553
3739992120,1cyclictest0-21swapper/309:02:523
3739992120,1cyclictest0-21swapper/308:27:533
373999210,20cyclictest0-21swapper/312:22:523
373999210,20cyclictest0-21swapper/311:42:543
373999210,20cyclictest0-21swapper/310:27:553
373999210,1cyclictest9769-21apt-get10:42:533
373999210,0cyclictest0-21swapper/309:57:523
373999210,0cyclictest0-21swapper/307:12:503
3738992121,0cyclictest4957-21apt-get10:32:542
3738992121,0cyclictest30070-21apt-get09:12:542
3738992121,0cyclictest1314-21apt-get11:30:022
3738992119,1cyclictest9443-21apt-get11:47:542
3738992118,2cyclictest21073-21cron07:45:002
373899210,20cyclictest12767-21perf08:35:002
373899210,20cyclictest0-21swapper/210:42:532
373899210,20cyclictest0-21swapper/208:27:532
373899210,0cyclictest1091-21apt-get12:37:542
373899210,0cyclictest0-21swapper/210:50:032
373899210,0cyclictest0-21swapper/210:12:512
3737992121,0cyclictest0-21swapper/108:42:541
3737992119,1cyclictest612-21gdbus12:05:001
3737992119,1cyclictest29261-21turbostat10:15:001
3737992119,1cyclictest0-21swapper/109:44:591
373799210,20cyclictest8398-21apt-get08:27:521
373799210,20cyclictest0-21swapper/108:07:531
373799210,20cyclictest0-21swapper/107:27:541
373799210,1cyclictest0-21swapper/109:35:001
373799210,0cyclictest0-21swapper/110:32:551
373799210,0cyclictest0-21swapper/108:52:531
373799210,0cyclictest0-21swapper/107:52:571
3736992120,1cyclictest31190-21apt-get08:07:540
3736992119,1cyclictest28308-21ls08:00:000
373699210,20cyclictest12218-21apt-get10:47:520
373699210,20cyclictest0-21swapper/008:32:530
373699210,0cyclictest0-21swapper/012:02:530
373699210,0cyclictest0-21swapper/011:37:520
373699210,0cyclictest0-21swapper/007:22:510
3739992019,0cyclictest0-21swapper/309:47:553
373999200,20cyclictest0-21swapper/308:57:543
373999200,20cyclictest0-21swapper/307:17:523
373999200,19cyclictest0-21swapper/308:02:513
373999200,0cyclictest0-21swapper/312:02:533
373999200,0cyclictest0-21swapper/311:47:543
373999200,0cyclictest0-21swapper/309:07:523
3738992020,0cyclictest0-21swapper/212:07:542
3738992020,0cyclictest0-21swapper/211:27:532
3738992020,0cyclictest0-21swapper/211:22:552
3738992020,0cyclictest0-21swapper/211:17:552
3738992020,0cyclictest0-21swapper/211:07:532
3738992020,0cyclictest0-21swapper/209:27:532
3738992020,0cyclictest0-21swapper/208:22:532
3738992020,0cyclictest0-21swapper/208:12:552
3738992019,1cyclictest28717-21apt-get12:27:552
3738992019,1cyclictest24948-21apt-get10:07:522
373899200,20cyclictest0-21swapper/209:52:522
373899200,0cyclictest6602-21apt-get07:17:522
373899200,0cyclictest0-21swapper/210:47:522
373899200,0cyclictest0-21swapper/210:17:532
373899200,0cyclictest0-21swapper/209:40:022
3737992020,0cyclictest3573-21apt-get08:17:521
3737992020,0cyclictest32469-21apt-get09:17:541
3737992020,0cyclictest0-21swapper/111:57:541
3737992020,0cyclictest0-21swapper/111:22:551
3737992020,0cyclictest0-21swapper/109:25:011
3737992020,0cyclictest0-21swapper/108:12:551
3737992020,0cyclictest0-21swapper/107:20:021
373799200,0cyclictest22858-21apt-get08:57:541
373799200,0cyclictest0-21swapper/112:12:561
373799200,0cyclictest0-21swapper/111:47:531
3736992020,0cyclictest0-21swapper/012:22:510
3736992020,0cyclictest0-21swapper/011:57:540
3736992020,0cyclictest0-21swapper/011:50:000
3736992020,0cyclictest0-21swapper/010:22:530
3736992020,0cyclictest0-21swapper/009:42:530
3736992020,0cyclictest0-21swapper/009:25:000
3736992020,0cyclictest0-21swapper/009:17:530
3736992020,0cyclictest0-21swapper/008:42:530
3736992020,0cyclictest0-21swapper/007:52:570
3736992020,0cyclictest0-21swapper/007:17:520
3736992019,1cyclictest0-21swapper/011:10:000
373699200,20cyclictest0-21swapper/011:02:550
373699200,20cyclictest0-21swapper/007:32:520
373699200,0cyclictest21583-21apt-get07:47:540
373699200,0cyclictest0-21swapper/009:47:550
373699200,0cyclictest0-21swapper/008:27:530
3739991919,0cyclictest0-21swapper/312:32:553
3739991919,0cyclictest0-21swapper/310:47:523
3739991919,0cyclictest0-21swapper/310:22:523
3739991919,0cyclictest0-21swapper/310:17:523
3739991919,0cyclictest0-21swapper/310:07:513
3739991919,0cyclictest0-21swapper/308:37:533
3739991919,0cyclictest0-21swapper/307:47:543
3739991917,1cyclictest25847-21perf12:20:003
3739991915,3cyclictest24479-21apt10:05:013
373999190,18cyclictest0-21swapper/309:22:533
3738991919,0cyclictest0-21swapper/212:17:532
3738991919,0cyclictest0-21swapper/212:02:522
3738991919,0cyclictest0-21swapper/211:37:532
3738991919,0cyclictest0-21swapper/209:02:532
3738991919,0cyclictest0-21swapper/208:47:542
3738991919,0cyclictest0-21swapper/208:37:532
3738991919,0cyclictest0-21swapper/208:17:522
3738991919,0cyclictest0-21swapper/207:47:532
3738991918,1cyclictest31112-21apt-get12:32:552
3738991918,1cyclictest0-21swapper/207:55:002
3738991917,1cyclictest24809-21perf09:00:002
3738991917,1cyclictest24455-21awk10:05:012
373899190,18cyclictest8954-21cron11:45:012
3737991919,0cyclictest31834-21apt-get11:27:531
3737991919,0cyclictest15344-21apt-get09:47:551
3737991919,0cyclictest0-21swapper/111:17:551
3737991919,0cyclictest0-21swapper/108:02:531
3737991919,0cyclictest0-21swapper/107:37:521
3737991919,0cyclictest0-21swapper/107:12:511
3737991918,1cyclictest2475-21apt-get09:22:521
3737991918,0cyclictest2548-21apt-get10:27:551
3737991917,1cyclictest2078-21munin-run10:25:001
373799190,18cyclictest8550-21munin-run07:20:001
3736991919,0cyclictest0-21swapper/012:12:560
3736991919,0cyclictest0-21swapper/011:22:550
3736991919,0cyclictest0-21swapper/009:25:000
3739991818,0cyclictest10849-21apt-get08:32:523
3739991818,0cyclictest0-21swapper/312:27:553
3739991818,0cyclictest0-21swapper/311:07:543
3739991818,0cyclictest0-21swapper/309:52:523
3739991818,0cyclictest0-21swapper/309:32:533
3739991818,0cyclictest0-21swapper/307:27:543
3739991817,1cyclictest23982-21apt-get07:52:573
3739991817,1cyclictest17410-21apt-get10:57:543
3739991816,1cyclictest28967-21munin-run11:19:593
3738991818,0cyclictest0-21swapper/211:02:542
3738991818,0cyclictest0-21swapper/209:57:522
3738991818,0cyclictest0-21swapper/209:32:532
3738991817,1cyclictest15664-21apt-get08:42:522
3738991816,1cyclictest16192-21perf07:35:002
3737991818,0cyclictest0-21swapper/111:52:541
3737991816,1cyclictest28253-21sleep12:25:001
3737991816,1cyclictest19654-21latency_hist09:55:001
373799180,17cyclictest0-21swapper/111:42:541
373799180,0cyclictest0-21swapper/108:37:531
3736991818,0cyclictest0-21swapper/011:27:530
3736991818,0cyclictest0-21swapper/010:07:520
3736991818,0cyclictest0-21swapper/009:52:520
3736991818,0cyclictest0-21swapper/009:12:540
3736991818,0cyclictest0-21swapper/009:02:530
3736991817,1cyclictest0-21swapper/010:42:530
3736991816,1cyclictest0-21swapper/008:25:000
3739991717,0cyclictest0-21swapper/311:02:553
3739991717,0cyclictest0-21swapper/307:37:533
3738991717,0cyclictest24621-21apt-get11:12:532
3738991717,0cyclictest0-21swapper/210:27:552
3738991717,0cyclictest0-21swapper/208:07:542
3738991717,0cyclictest0-21swapper/207:57:542
3738991717,0cyclictest0-21swapper/207:37:532
373899170,16cyclictest19312-21sh11:00:002
3737991717,0cyclictest25274-21apt-get09:02:531
3737991717,0cyclictest19990-21perf08:50:001
3737991717,0cyclictest0-21swapper/112:37:551
3737991716,0cyclictest4625-21apt-get11:37:531
3736991717,0cyclictest14139-21apt-get10:50:030
3736991717,0cyclictest0-21swapper/011:32:530
3736991717,0cyclictest0-21swapper/011:12:530
3736991717,0cyclictest0-21swapper/009:57:520
3736991717,0cyclictest0-21swapper/008:47:550
3736991717,0cyclictest0-21swapper/007:27:530
3736991716,0cyclictest0-21swapper/012:17:530
373699170,0cyclictest0-21swapper/011:52:540
3739991616,0cyclictest0-21swapper/312:12:543
3739991616,0cyclictest0-21swapper/308:42:533
3739991616,0cyclictest0-21swapper/308:22:533
3739991615,1cyclictest0-21swapper/307:42:513
3739991614,1cyclictest29272-21munin-plugin-st10:14:593
3738991616,0cyclictest20460-21apt-get08:52:522
3738991614,1cyclictest0-21swapper/207:12:582
373899160,15cyclictest0-21swapper/209:35:542
3737991616,0cyclictest6001-21apt-get08:22:531
3737991616,0cyclictest26374-21apt-get07:57:531
3737991616,0cyclictest0-21swapper/109:12:541
3736991616,0cyclictest0-21swapper/010:57:540
3736991616,0cyclictest0-21swapper/007:42:520
3736991613,2cyclictest9606-21perf09:34:590
3736991613,0cyclictest0-21swapper/007:14:180
373699160,0cyclictest0-21swapper/010:32:540
3739991515,0cyclictest0-21swapper/309:37:533
3739991515,0cyclictest0-21swapper/309:17:533
3738991513,1cyclictest313-21systemd-journal12:15:002
3738991513,1cyclictest0-21swapper/209:10:002
373899150,1cyclictest578-21irqbalance11:57:382
373899150,1cyclictest468-21cstates10:22:532
373899150,1cyclictest27713-21awk12:23:002
373899150,14cyclictest0-21swapper/211:52:552
3737991515,0cyclictest23900-21apt-get12:17:531
3737991515,0cyclictest0-21swapper/109:37:531
3736991515,0cyclictest0-21swapper/012:27:550
373699150,14cyclictest22876-21cstates10:02:530
373699150,14cyclictest0-21swapper/007:37:540
3739991414,0cyclictest0-21swapper/309:25:013
3739991414,0cyclictest0-21swapper/307:34:073
373999140,14cyclictest0-21swapper/311:50:213
373999140,14cyclictest0-21swapper/311:26:023
373999140,14cyclictest0-21swapper/308:46:043
373999140,14cyclictest0-21swapper/308:07:233
373999140,0cyclictest0-21swapper/311:10:433
373999140,0cyclictest0-21swapper/308:52:183
3738991414,0cyclictest0-21swapper/210:35:232
373899140,14cyclictest0-21swapper/207:26:282
373899140,0cyclictest0-21swapper/208:00:542
373799140,14cyclictest0-21swapper/112:05:131
373799140,14cyclictest0-21swapper/110:47:311
3736991414,0cyclictest0-21swapper/010:25:160
3736991414,0cyclictest0-21swapper/008:35:030
373699140,14cyclictest0-21swapper/012:05:540
373699140,14cyclictest0-21swapper/010:11:030
373699140,0cyclictest0-21swapper/008:50:230
373699140,0cyclictest0-21swapper/008:00:190
3739991313,0cyclictest0-21swapper/310:30:273
3736991313,0cyclictest0-21swapper/008:12:550
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional