You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-12 - 17:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Fri Dec 12, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2993999270,20cyclictest0-21swapper/021:33:550
29944992523,1cyclictest31683-21munin-run20:20:001
29939992523,1cyclictest21311-21sh21:05:000
29950992421,2cyclictest313-21systemd-journal19:30:003
29948992422,1cyclictest16499-21cron20:55:012
29939992420,4cyclictest0-21swapper/019:33:520
29939992419,4cyclictest15253-21apt-get23:08:520
2993999240,23cyclictest28546-21perf21:20:000
29950992321,1cyclictest0-21swapper/321:05:003
2995099230,22cyclictest313-21systemd-journal23:20:003
2995099230,22cyclictest0-21swapper/319:48:523
29948992321,1cyclictest8846-21sh20:40:002
29948992321,1cyclictest8846-21sh20:40:002
29948992321,1cyclictest17243-21cat19:50:002
29948992321,1cyclictest16769-21cron00:15:012
2994899230,22cyclictest23566-21apt-get22:18:532
2994899230,22cyclictest18909-21perf21:00:002
2994899230,22cyclictest17096-21awk23:10:002
2994899230,0cyclictest0-21swapper/222:03:532
2994499230,22cyclictest593-21dbus-daemon20:35:001
29939992317,5cyclictest31891-21sendmail23:40:000
2993999230,2cyclictest19477-21basename23:15:000
29950992220,1cyclictest12955-21munin-run21:55:003
29950992220,1cyclictest0-21swapper/300:15:013
2995099220,21cyclictest30223-21latency_hist22:30:003
2995099220,21cyclictest2322-21munin-plugin-st23:45:003
2995099220,21cyclictest12226-21apt-get20:48:523
2995099220,21cyclictest0-21swapper/322:25:003
29948992221,1cyclictest22469-21apt-get23:23:572
29948992220,1cyclictest10564-21munin-run21:50:012
29948992219,2cyclictest313-21systemd-journal21:35:002
2994899220,21cyclictest422-21apt-get19:18:522
2994899220,21cyclictest20047-21apt-get23:18:522
2994899220,21cyclictest12936-21ls21:55:002
2994899220,1cyclictest9829-21apt-get20:43:532
2994899220,1cyclictest19490-21apt-get21:03:532
2994899220,1cyclictest0-21swapper/223:40:012
29944992220,1cyclictest11682-21munin-run20:45:001
2994499220,21cyclictest31902-21idleruntime-cro23:39:591
2994499220,21cyclictest24309-21perf23:25:011
2994499220,21cyclictest0-21swapper/122:58:531
2994499220,21cyclictest0-21swapper/122:03:521
2994499220,1cyclictest23005-21munin-plugin-st22:15:001
29939992221,1cyclictest0-21swapper/022:33:530
29939992220,1cyclictest17358-21apt-get00:18:530
29939992220,1cyclictest0-21swapper/020:40:020
2993999220,21cyclictest6315-21apt-get21:43:530
2993999220,21cyclictest5012-21apt-get19:25:320
2993999220,21cyclictest0-21swapper/020:33:520
2993999220,21cyclictest0-21swapper/020:03:520
2993999220,21cyclictest0-21swapper/019:58:520
2993999220,21cyclictest0-21swapper/019:43:530
2993999220,21cyclictest0-21swapper/019:38:530
2993999220,1cyclictest0-21swapper/020:13:530
29950992120,1cyclictest10167-21apt-get00:03:533
29950992119,1cyclictest0-21swapper/323:55:003
29950992119,1cyclictest0-21swapper/321:45:013
2995099210,21cyclictest0-21swapper/322:10:023
2995099210,20cyclictest25005-21apt-get20:08:523
2995099210,20cyclictest0-21swapper/322:43:533
2995099210,20cyclictest0-21swapper/321:58:543
2995099210,0cyclictest10386-21apt-get22:58:523
2995099210,0cyclictest0-21swapper/300:33:553
29948992120,1cyclictest0-21swapper/219:28:522
29948992119,1cyclictest17648-21apt-get23:13:532
2994899210,20cyclictest7989-21apt-get22:53:532
2994899210,20cyclictest27360-21apt-get00:38:552
2994899210,20cyclictest20206-21apt-get22:10:002
2994899210,20cyclictest0-21swapper/223:53:562
2994899210,1cyclictest7439-21perf22:49:592
2994899210,0cyclictest10135-21apt-get19:38:532
2994899210,0cyclictest0-21swapper/223:58:542
2994899210,0cyclictest0-21swapper/220:18:512
29944992120,0cyclictest0-21swapper/123:03:571
29944992119,1cyclictest7420-21sh22:50:001
29944992119,1cyclictest12020-21munin-run00:05:001
2994499210,20cyclictest25986-21apt-get22:23:561
2994499210,20cyclictest0-21swapper/119:53:551
2994499210,20cyclictest0-21swapper/100:28:561
2994499210,0cyclictest24964-21apt-get00:33:541
2994499210,0cyclictest0-21swapper/122:53:531
29939992121,0cyclictest0-21swapper/023:23:570
29939992121,0cyclictest0-21swapper/022:18:540
29939992121,0cyclictest0-21swapper/019:48:510
29939992120,1cyclictest19648-21perf19:55:000
29939992120,1cyclictest0-21swapper/021:23:550
29939992119,1cyclictest0-21swapper/020:25:000
2993999210,21cyclictest0-21swapper/000:33:550
2993999210,20cyclictest0-21swapper/023:48:520
2993999210,20cyclictest0-21swapper/020:38:530
2993999210,20cyclictest0-21swapper/020:38:530
2993999210,0cyclictest0-21swapper/023:18:520
2993999210,0cyclictest0-21swapper/021:53:550
29950992020,0cyclictest0-21swapper/322:33:533
29950992020,0cyclictest0-21swapper/320:33:523
29950992020,0cyclictest0-21swapper/320:28:523
29950992020,0cyclictest0-21swapper/320:23:543
29950992019,0cyclictest24862-21apt-get23:28:543
29950992018,1cyclictest612-21gdbus22:50:003
29950992018,1cyclictest0-21swapper/323:50:003
29950992018,1cyclictest0-21swapper/322:20:003
2995099200,0cyclictest2893-21apt-get19:23:533
2995099200,0cyclictest0-21swapper/320:58:553
2995099200,0cyclictest0-21swapper/300:25:013
29948992020,0cyclictest21890-21apt-get21:08:562
29948992020,0cyclictest0-21swapper/222:58:532
29948992020,0cyclictest0-21swapper/221:13:512
29948992020,0cyclictest0-21swapper/220:08:522
29948992020,0cyclictest0-21swapper/200:23:542
29948992019,1cyclictest30062-21apt-get23:38:542
29948992019,0cyclictest0-21swapper/221:18:552
29948992018,1cyclictest19636-21latency_hist19:55:002
2994899200,20cyclictest0-21swapper/221:23:552
2994899200,0cyclictest30787-21apt-get22:33:532
2994899200,0cyclictest0-21swapper/200:18:542
29944992020,0cyclictest13508-21apt-get21:58:541
29944992020,0cyclictest0-21swapper/120:23:541
29944992019,1cyclictest0-21swapper/120:53:531
29944992018,1cyclictest8715-21apt-get21:48:551
29944992018,1cyclictest11995-21perf19:40:001
29944992018,1cyclictest0-21swapper/123:28:541
2994499200,20cyclictest0-21swapper/123:08:531
2994499200,19cyclictest22610-21apt-get20:03:521
2994499200,0cyclictest0-21swapper/120:38:541
2994499200,0cyclictest0-21swapper/120:38:541
29939992020,0cyclictest0-21swapper/023:58:540
29939992020,0cyclictest0-21swapper/022:53:530
2993999200,20cyclictest0-21swapper/023:33:570
2993999200,0cyclictest0-21swapper/022:10:020
2993999200,0cyclictest0-21swapper/021:08:560
29950991919,0cyclictest0-21swapper/323:13:533
29950991919,0cyclictest0-21swapper/322:08:513
29950991919,0cyclictest0-21swapper/319:18:523
29950991918,1cyclictest9834-21perf22:55:003
29950991918,1cyclictest0-21swapper/320:53:543
29950991917,1cyclictest0-21swapper/322:05:003
29950991917,1cyclictest0-21swapper/320:40:023
29948991919,0cyclictest0-21swapper/222:13:552
29948991919,0cyclictest0-21swapper/221:29:282
29948991919,0cyclictest0-21swapper/200:03:532
29948991917,1cyclictest9571-21turbostat19:35:002
29948991917,1cyclictest753-21apt-get22:38:572
29948991917,1cyclictest593-21dbus-daemon23:50:002
29944991919,0cyclictest0-21swapper/122:18:541
29944991919,0cyclictest0-21swapper/122:08:501
29944991919,0cyclictest0-21swapper/119:18:521
29944991919,0cyclictest0-21swapper/119:13:521
29944991918,0cyclictest31906-21apt-get23:40:011
2994499190,0cyclictest0-21swapper/121:23:551
29939991919,0cyclictest0-21swapper/022:38:570
29939991919,0cyclictest0-21swapper/022:28:570
29939991919,0cyclictest0-21swapper/000:23:530
29950991818,0cyclictest0-21swapper/323:58:553
29950991818,0cyclictest0-21swapper/322:38:573
29950991818,0cyclictest0-21swapper/320:13:533
29950991818,0cyclictest0-21swapper/319:58:523
29950991818,0cyclictest0-21swapper/300:23:543
29950991818,0cyclictest0-21swapper/300:18:543
29950991815,2cyclictest313-21systemd-journal23:25:013
29948991818,0cyclictest0-21swapper/223:33:562
29948991818,0cyclictest0-21swapper/222:23:562
29948991818,0cyclictest0-21swapper/221:43:532
29948991817,1cyclictest4615-21apt-get20:33:512
29948991817,1cyclictest0-21swapper/220:28:532
29944991818,0cyclictest0-21swapper/122:28:571
29944991818,0cyclictest0-21swapper/121:38:581
29944991818,0cyclictest0-21swapper/121:08:561
29944991818,0cyclictest0-21swapper/100:18:541
29939991818,0cyclictest0-21swapper/022:58:530
29939991818,0cyclictest0-21swapper/020:20:000
29939991818,0cyclictest0-21swapper/000:38:550
29939991818,0cyclictest0-21swapper/000:13:530
29939991817,0cyclictest5267-21apt-get23:53:560
29939991816,1cyclictest3903-21apt-get21:38:580
29939991816,1cyclictest0-21swapper/022:00:000
29939991814,3cyclictest0-21swapper/000:29:000
29950991717,0cyclictest27661-21apt-get23:33:573
29950991717,0cyclictest0-21swapper/320:03:523
29950991717,0cyclictest0-21swapper/300:08:543
2995099170,16cyclictest5720-21cron21:40:003
29948991717,0cyclictest0-21swapper/220:03:522
29948991716,1cyclictest0-21swapper/221:38:582
29948991716,0cyclictest0-21swapper/222:43:532
29944991717,0cyclictest12564-21apt-get00:08:541
29944991717,0cyclictest0-21swapper/122:33:541
29944991717,0cyclictest0-21swapper/121:53:561
29944991717,0cyclictest0-21swapper/121:28:531
29944991717,0cyclictest0-21swapper/120:13:531
29944991717,0cyclictest0-21swapper/119:48:521
29944991716,1cyclictest0-21swapper/121:33:551
29944991714,2cyclictest9472-40mandb00:00:001
29944991714,2cyclictest16810-21dump-pmu-power00:15:011
29939991717,0cyclictest0-21swapper/023:03:570
29939991717,0cyclictest0-21swapper/022:08:500
29939991717,0cyclictest0-21swapper/021:13:520
29939991717,0cyclictest0-21swapper/000:03:530
29939991715,2cyclictest4926-21awk22:44:040
29939991715,1cyclictest0-21swapper/020:55:010
2993999170,16cyclictest25498-21sed23:28:560
29950991616,0cyclictest0-21swapper/323:08:533
29950991616,0cyclictest0-21swapper/321:48:553
29950991616,0cyclictest0-21swapper/321:08:553
29950991616,0cyclictest0-21swapper/320:18:513
29950991614,1cyclictest28512-21sh21:20:003
29948991616,0cyclictest0-21swapper/219:40:012
29948991615,1cyclictest12808-21apt-get23:03:572
29948991615,1cyclictest0-21swapper/220:13:532
29948991614,1cyclictest26479-21grep23:29:042
29944991616,0cyclictest17076-21apt-get20:58:541
29944991616,0cyclictest0-21swapper/123:48:531
29944991616,0cyclictest0-21swapper/123:18:521
29944991616,0cyclictest0-21swapper/121:18:551
29944991616,0cyclictest0-21swapper/119:58:521
29944991615,0cyclictest24283-21apt-get21:13:521
29939991616,0cyclictest0-21swapper/023:43:550
29939991616,0cyclictest0-21swapper/019:23:530
29939991615,1cyclictest0-21swapper/019:13:520
29939991612,3cyclictest2847-21tail20:28:550
2993999160,16cyclictest0-21swapper/021:48:550
2993999160,16cyclictest0-21swapper/020:09:090
29950991515,0cyclictest0-21swapper/319:40:023
29950991515,0cyclictest0-21swapper/319:38:533
29948991515,0cyclictest0-21swapper/219:23:522
2994899150,1cyclictest0-21swapper/220:24:022
29944991515,0cyclictest7737-21apt-get19:33:521
29944991514,1cyclictest5855-21fschecks_time19:28:541
29944991513,1cyclictest19499-21munin-plugin-st23:15:001
29939991515,0cyclictest0-21swapper/021:28:530
29939991515,0cyclictest0-21swapper/020:48:510
2993999150,15cyclictest0-21swapper/020:56:360
2993999150,14cyclictest0-21swapper/022:49:000
29950991414,0cyclictest0-21swapper/319:51:013
2995099140,14cyclictest0-21swapper/323:38:383
2995099140,14cyclictest0-21swapper/323:01:163
2995099140,14cyclictest0-21swapper/321:32:213
2995099140,14cyclictest0-21swapper/321:25:023
2995099140,14cyclictest0-21swapper/321:21:183
2995099140,14cyclictest0-21swapper/321:10:413
2995099140,14cyclictest0-21swapper/320:35:573
2995099140,14cyclictest0-21swapper/320:35:563
2995099140,0cyclictest0-21swapper/319:31:143
2995099140,0cyclictest0-21swapper/300:35:253
29948991414,0cyclictest0-21swapper/200:26:092
2994899140,14cyclictest0-21swapper/221:57:372
2994899140,14cyclictest0-21swapper/220:45:322
2994899140,14cyclictest0-21swapper/219:55:262
2994899140,14cyclictest0-21swapper/219:12:142
2994899140,14cyclictest0-21swapper/200:32:492
2994899140,14cyclictest0-21swapper/200:06:032
2994899140,0cyclictest0-21swapper/222:25:472
29944991414,0cyclictest0-21swapper/122:41:311
29944991414,0cyclictest0-21swapper/119:40:561
2994499140,14cyclictest0-21swapper/123:52:311
2994499140,14cyclictest0-21swapper/120:45:011
2994499140,14cyclictest0-21swapper/119:20:361
2994499140,14cyclictest0-21swapper/100:38:361
2994499140,14cyclictest0-21swapper/100:20:161
2994499140,0cyclictest0-21swapper/121:40:131
2994499140,0cyclictest0-21swapper/121:02:541
29939991414,0cyclictest0-21swapper/000:05:330
2993999140,14cyclictest0-21swapper/022:02:310
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional