You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-18 - 04:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Tue Aug 18, 2026 00:45:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1872047997920,55cyclictest12-21ksoftirqd/000:38:180
1872047996529,5cyclictest12-21ksoftirqd/023:01:430
1872047996417,3cyclictest131rcu_preempt21:13:040
1872047996321,38cyclictest12-21ksoftirqd/022:33:100
187204799613,18cyclictest1949300-21sh23:40:000
1872047996128,4cyclictest12-21ksoftirqd/023:24:530
1872047996128,4cyclictest12-21ksoftirqd/023:24:530
1872047996120,36cyclictest16850irq/134-800000022:40:020
1872047996118,3cyclictest131rcu_preempt21:48:290
1872047996117,40cyclictest12-21ksoftirqd/022:36:370
1872047996022,18cyclictest48-21kcompactd000:09:250
1872047995929,4cyclictest131rcu_preempt22:04:300
187204799591,25cyclictest131rcu_preempt23:27:060
1872060995850,6cyclictest37-21ksoftirqd/321:00:023
1872060995850,6cyclictest37-21ksoftirqd/321:00:013
1872055995849,6cyclictest1883076-21kworker/2:222:00:212
187204799589,18cyclictest0-21swapper/023:17:380
1872047995829,14cyclictest1946584-21ssh23:34:070
187204799579,3cyclictest131rcu_preempt21:37:580
1872047995734,19cyclictest12-21ksoftirqd/020:05:010
1872047995734,19cyclictest12-21ksoftirqd/020:05:000
1872047995729,13cyclictest1974788-21ssh00:33:230
187204799571,37cyclictest131rcu_preempt23:44:220
1872055995642,6cyclictest1861318-21kworker/2:122:15:002
1872055995642,6cyclictest1861318-21kworker/2:122:14:592
1872047995628,13cyclictest1887375-21ssh21:29:120
1872055995515,7cyclictest0-21swapper/223:34:482
1872054995538,8cyclictest0-21swapper/100:00:031
1872054995538,8cyclictest0-21swapper/100:00:021
1872054995528,6cyclictest23-21ksoftirqd/120:00:001
1872047995528,12cyclictest1964960-21ssh00:12:330
1872047995527,12cyclictest1901011-21ssh21:57:490
1872047995518,32cyclictest16850irq/134-800000000:01:490
1872047995517,34cyclictest16850irq/134-800000022:55:010
1872047995514,37cyclictest12-21ksoftirqd/022:23:320
1872055995445,5cyclictest1861318-21kworker/2:123:05:012
1872055995444,6cyclictest1861318-21kworker/2:121:15:032
1872055995440,5cyclictest1861318-21kworker/2:121:20:202
187205599541,3cyclictest131rcu_preempt20:38:002
1872047995419,5cyclictest16850irq/134-800000022:29:580
1872047995419,31cyclictest16850irq/134-800000021:34:100
1872047995418,32cyclictest16850irq/134-800000021:42:570
1872047995417,11cyclictest1899629-21munin-run21:55:010
1872047995417,11cyclictest1899629-21munin-run21:55:000
187204799541,38cyclictest1923702-21ssh22:45:420
187204799541,37cyclictest1969244-21ssh00:21:250
1872060995330,17cyclictest275-21dbus-daemon00:30:013
1872055995335,9cyclictest1861318-21kworker/2:121:35:532
1872047995324,14cyclictest1956353-21ssh23:54:430
1872047995318,32cyclictest16850irq/134-800000022:58:060
1872047995318,31cyclictest16850irq/134-800000023:47:130
1872047995318,31cyclictest16850irq/134-800000022:13:420
1872047995318,31cyclictest16850irq/134-800000022:13:420
1872047995318,30cyclictest16850irq/134-800000023:11:340
1872047995317,31cyclictest16850irq/134-800000000:19:170
187204799531,29cyclictest131rcu_preempt23:55:410
187204799531,29cyclictest131rcu_preempt23:55:410
1872055995246,3cyclictest1873584-21kworker/2:220:00:032
1872055995244,5cyclictest1925623-21kworker/2:000:08:152
1872055995244,5cyclictest1861318-21kworker/2:121:16:072
1872055995243,6cyclictest1883076-21kworker/2:221:58:062
1872055995243,6cyclictest1861318-21kworker/2:122:52:172
1872047995218,30cyclictest16850irq/134-800000000:25:380
1872047995217,31cyclictest16850irq/134-800000022:18:120
1872047995216,31cyclictest16850irq/134-800000021:15:060
1872055995143,5cyclictest1883076-21kworker/2:221:53:522
1872055995143,5cyclictest1883076-21kworker/2:221:53:522
1872055995143,5cyclictest1861318-21kworker/2:122:45:192
1872055995143,5cyclictest1861318-21kworker/2:121:00:022
1872055995143,5cyclictest1861318-21kworker/2:121:00:022
1872055995142,5cyclictest1861318-21kworker/2:121:32:172
1872054995130,4cyclictest23-21ksoftirqd/122:35:011
1872054995123,20cyclictest23-21ksoftirqd/120:37:061
1872054995122,20cyclictest23-21ksoftirqd/100:36:141
1872047995142,5cyclictest16950irq/224-800000020:55:010
187204799512,45cyclictest1872247-21munin-run19:15:000
1872047995116,30cyclictest16850irq/134-800000022:09:300
1872047995115,31cyclictest16850irq/134-800000023:09:300
187204799511,3cyclictest131rcu_preempt19:25:020
1872060995039,6cyclictest1883033-21sendmail21:20:013
1872055995042,5cyclictest1954150-21kworker/2:300:27:132
1872055995042,5cyclictest1925623-21kworker/2:023:41:132
1872055995042,5cyclictest1925623-21kworker/2:023:19:032
1872055995042,5cyclictest1925623-21kworker/2:000:12:182
1872055995042,5cyclictest1861318-21kworker/2:122:36:242
1872055995042,5cyclictest1861318-21kworker/2:121:27:512
1872055995041,6cyclictest1861318-21kworker/2:121:41:212
1872055995040,6cyclictest1954150-21kworker/2:300:19:482
1872055995040,6cyclictest1883076-21kworker/2:221:46:362
1872055995028,13cyclictest30-21ksoftirqd/223:25:022
1872055995028,13cyclictest30-21ksoftirqd/223:25:012
1872054995020,22cyclictest23-21ksoftirqd/100:10:291
187206099497,6cyclictest131rcu_preempt23:25:023
187206099497,6cyclictest131rcu_preempt23:25:013
1872060994910,6cyclictest131rcu_preempt22:28:313
1872060994910,34cyclictest1-21systemd21:15:023
1872055994941,5cyclictest1954150-21kworker/2:300:24:222
1872055994941,5cyclictest1861318-21kworker/2:122:20:022
1872055994940,6cyclictest1861318-21kworker/2:122:59:402
1872055994932,12cyclictest3467602-21systemd-journal23:59:592
1872055994932,12cyclictest3467602-21systemd-journal00:00:002
1872054994922,23cyclictest23-21ksoftirqd/123:00:051
1872054994919,22cyclictest23-21ksoftirqd/121:30:021
1872047994914,31cyclictest16850irq/134-800000021:21:310
1872055994840,5cyclictest1925623-21kworker/2:023:50:062
1872055994840,5cyclictest1861318-21kworker/2:123:09:322
187205499482,5cyclictest131rcu_preempt21:15:011
1872054994825,15cyclictest23-21ksoftirqd/120:45:031
1872054994823,6cyclictest23-21ksoftirqd/121:00:021
1872054994823,6cyclictest23-21ksoftirqd/121:00:011
1872054994815,24cyclictest23-21ksoftirqd/121:52:031
1872054994815,24cyclictest23-21ksoftirqd/121:52:031
187205499481,12cyclictest1885482-21cat21:25:011
1872060994714,10cyclictest37-21ksoftirqd/322:32:193
1872055994739,5cyclictest1861318-21kworker/2:122:32:162
1872055994739,5cyclictest1861318-21kworker/2:121:05:002
1872055994738,6cyclictest1925623-21kworker/2:023:29:412
1872055994731,7cyclictest1883076-21kworker/2:222:06:042
187205499473,9cyclictest23-21ksoftirqd/100:05:011
1872054994737,6cyclictest1878280-21diskmemload00:20:011
1872054994731,13cyclictest3467602-21systemd-journal19:12:381
1872054994729,14cyclictest1-21systemd21:10:011
1872054994724,6cyclictest23-21ksoftirqd/121:50:001
1872054994723,5cyclictest23-21ksoftirqd/119:30:001
1872054994723,5cyclictest23-21ksoftirqd/100:04:201
1872054994722,6cyclictest23-21ksoftirqd/122:05:011
187205499471,9cyclictest1921066-21latency_hist22:40:021
187204799472,13cyclictest1874553-21sshd19:57:390
1872060994633,9cyclictest3467602-21systemd-journal22:43:383
1872060994625,16cyclictest275-21dbus-daemon22:15:013
1872060994625,16cyclictest275-21dbus-daemon22:15:013
1872060994620,16cyclictest37-21ksoftirqd/300:40:013
1872055994639,4cyclictest1861318-21kworker/2:123:10:402
1872055994638,5cyclictest1954150-21kworker/2:300:36:112
1872055994638,5cyclictest1873584-21kworker/2:220:20:012
1872055994638,5cyclictest1861318-21kworker/2:122:26:582
1872055994629,7cyclictest1954150-21kworker/2:300:03:202
1872054994625,13cyclictest275-21dbus-daemon20:55:011
1872054994622,5cyclictest23-21ksoftirqd/123:49:461
1872054994622,5cyclictest23-21ksoftirqd/123:45:021
1872047994633,10cyclictest3467602-21systemd-journal20:10:020
187204799461,23cyclictest131rcu_preempt21:05:000
1872060994532,10cyclictest1-21systemd22:20:003
1872060994520,6cyclictest37-21ksoftirqd/321:58:303
1872060994520,5cyclictest37-21ksoftirqd/321:49:173
1872055994537,5cyclictest1954150-21kworker/2:300:33:392
1872055994537,5cyclictest1861318-21kworker/2:121:08:292
1872055994537,5cyclictest1861318-21kworker/2:120:47:562
1872054994527,14cyclictest275-21dbus-daemon22:15:011
1872054994527,14cyclictest275-21dbus-daemon22:15:001
1872054994525,5cyclictest23-21ksoftirqd/122:05:031
1872054994524,4cyclictest18879531ssh21:30:071
1872054994522,5cyclictest23-21ksoftirqd/122:41:451
1872054994520,5cyclictest23-21ksoftirqd/122:20:011
1872054994520,5cyclictest23-21ksoftirqd/121:44:071
1872047994520,5cyclictest12-21ksoftirqd/020:10:000
187206099446,5cyclictest131rcu_preempt23:37:133
1872060994430,11cyclictest1-21systemd20:45:003
187206099442,4cyclictest131rcu_preempt23:45:023
1872060994422,4cyclictest37-21ksoftirqd/323:00:003
1872060994420,4cyclictest37-21ksoftirqd/321:34:593
1872060994419,10cyclictest0-21swapper/322:55:023
1872055994436,5cyclictest1925623-21kworker/2:023:50:312
1872055994436,5cyclictest1873584-21kworker/2:220:13:362
1872055994436,5cyclictest1861318-21kworker/2:122:41:352
1872055994435,5cyclictest1861318-21kworker/2:122:23:192
1872055994431,10cyclictest1-21systemd19:44:592
1872054994434,7cyclictest1874309-21sshd19:54:391
1872054994433,8cyclictest1947449-21sshd23:35:381
1872054994423,4cyclictest23-21ksoftirqd/119:40:011
1872054994422,6cyclictest23-21ksoftirqd/123:06:341
1872054994422,4cyclictest23-21ksoftirqd/123:14:201
1872054994422,16cyclictest275-21dbus-daemon22:30:011
1872054994420,15cyclictest275-21dbus-daemon20:20:021
187204799443,3cyclictest131rcu_preempt19:45:010
1872047994432,9cyclictest1875612-21cron20:20:010
1872047994429,12cyclictest1878243-21sshd21:05:390
1872047994429,11cyclictest275-21dbus-daemon20:38:000
187204799442,37cyclictest364-21snmpd20:41:380
1872060994328,11cyclictest37-21ksoftirqd/300:09:053
1872060994320,5cyclictest37-21ksoftirqd/323:04:173
1872060994320,4cyclictest37-21ksoftirqd/321:45:023
1872055994334,6cyclictest1925623-21kworker/2:023:36:382
1872055994330,10cyclictest3467602-21systemd-journal20:05:022
1872055994330,10cyclictest3467602-21systemd-journal20:05:012
1872054994329,11cyclictest1883149-21cut21:20:021
187205499432,3cyclictest131rcu_preempt23:18:201
1872054994320,16cyclictest275-21dbus-daemon20:30:011
1872047994319,6cyclictest12-21ksoftirqd/019:50:370
187206099429,5cyclictest131rcu_preempt00:23:433
187206099424,3cyclictest131rcu_preempt00:18:233
187206099423,3cyclictest131rcu_preempt23:15:193
187206099422,3cyclictest131rcu_preempt21:25:023
1872060994222,4cyclictest37-21ksoftirqd/321:36:063
1872060994219,5cyclictest37-21ksoftirqd/322:35:303
187206099421,22cyclictest0-21swapper/323:30:223
1872054994229,10cyclictest1-21systemd20:49:591
1872054994227,11cyclictest1975478-21cron00:35:001
1872054994218,18cyclictest275-21dbus-daemon20:10:001
187205499421,5cyclictest131rcu_preempt23:30:191
187206099415,3cyclictest131rcu_preempt19:25:023
187206099414,3cyclictest150-21pr/ttyS223:55:003
187206099414,3cyclictest131rcu_preempt00:15:013
1872060994133,5cyclictest1872243-21latency_hist19:15:013
1872060994126,11cyclictest3467602-21systemd-journal23:07:353
1872060994120,15cyclictest275-21dbus-daemon19:50:013
1872060994120,15cyclictest275-21dbus-daemon19:50:003
1872060994119,5cyclictest37-21ksoftirqd/320:10:003
187206099411,4cyclictest131rcu_preempt20:00:003
1872055994130,8cyclictest1877004-21pluginstate20:45:022
1872055994127,11cyclictest1-21systemd20:25:012
1872055994125,12cyclictest275-21dbus-daemon19:40:002
187205499413,6cyclictest131rcu_preempt23:54:111
1872054994124,8cyclictest1892450-21modprobe21:40:011
1872054994117,5cyclictest23-21ksoftirqd/122:53:491
187205499411,3cyclictest131rcu_preempt22:55:051
1872047994123,14cyclictest1873273-21sshd19:34:390
1872047994118,4cyclictest12-21ksoftirqd/020:30:000
1872047994112,25cyclictest16950irq/224-800000020:50:000
187204799411,17cyclictest131rcu_preempt19:20:060
187206099405,3cyclictest131rcu_preempt00:00:003
187206099405,3cyclictest131rcu_preempt00:00:003
187206099404,4cyclictest131rcu_preempt23:26:593
187206099404,3cyclictest131rcu_preempt19:19:593
1872060994029,8cyclictest1-21systemd20:05:013
1872060994029,8cyclictest1-21systemd20:05:003
187206099402,5cyclictest131rcu_preempt00:34:593
1872060994025,11cyclictest1914059-21date22:25:033
1872060994023,14cyclictest1-21systemd21:55:023
1872060994023,14cyclictest1-21systemd21:55:023
1872060994020,5cyclictest37-21ksoftirqd/320:35:013
1872060994020,3cyclictest37-21ksoftirqd/319:55:013
187205599404,7cyclictest30-21ksoftirqd/219:15:022
1872055994028,9cyclictest30-21ksoftirqd/220:30:002
1872055994027,9cyclictest30-21ksoftirqd/220:35:012
187205499404,3cyclictest131rcu_preempt22:45:171
1872054994031,6cyclictest23-21ksoftirqd/100:27:001
187205499401,5cyclictest131rcu_preempt19:10:021
187204799404,3cyclictest131rcu_preempt19:50:020
187204799404,3cyclictest131rcu_preempt19:50:020
1872047994021,13cyclictest275-21dbus-daemon19:40:010
1872060993927,10cyclictest37-21ksoftirqd/323:13:193
1872060993925,11cyclictest37-21ksoftirqd/322:47:383
187206099392,4cyclictest131rcu_preempt22:03:373
1872060993921,13cyclictest1-21systemd20:50:003
1872060993920,14cyclictest275-21dbus-daemon19:35:003
1872060993919,5cyclictest37-21ksoftirqd/320:11:383
1872060993918,5cyclictest37-21ksoftirqd/321:25:553
1872060993918,5cyclictest37-21ksoftirqd/320:17:023
1872060993918,4cyclictest37-21ksoftirqd/300:01:393
1872055993931,5cyclictest1861318-21kworker/2:119:35:012
1872055993931,4cyclictest1876423-21kworker/2:320:54:592
187205499394,3cyclictest131rcu_preempt00:20:101
1872054993928,8cyclictest23-21ksoftirqd/122:23:161
1872054993928,8cyclictest1942229-21cron23:25:011
1872054993928,8cyclictest1942229-21cron23:25:011
1872054993928,8cyclictest1873273-21sshd19:34:381
1872054993920,5cyclictest23-21ksoftirqd/121:05:011
187205499391,3cyclictest131rcu_preempt23:27:401
1872047993929,7cyclictest0-21swapper/020:23:220
1872060993825,10cyclictest3467602-21systemd-journal20:39:393
1872060993819,4cyclictest37-21ksoftirqd/320:26:403
1872060993819,13cyclictest275-21dbus-daemon20:25:013
1872060993817,6cyclictest37-21ksoftirqd/322:05:293
187206099381,3cyclictest131rcu_preempt23:48:413
187206099381,3cyclictest131rcu_preempt21:05:013
1872055993831,5cyclictest1873584-21kworker/2:219:52:432
1872055993830,5cyclictest1861318-21kworker/2:119:27:382
1872055993810,13cyclictest0-21swapper/219:47:382
1872055993810,13cyclictest0-21swapper/219:47:372
1872054993829,6cyclictest23-21ksoftirqd/119:25:021
1872054993820,3cyclictest23-21ksoftirqd/119:20:011
187204799386,24cyclictest1872051-21latency_hist19:10:020
187204799381,34cyclictest0-21swapper/020:35:010
187206099374,4cyclictest131rcu_preempt19:45:003
1872055993725,9cyclictest30-21ksoftirqd/219:15:382
1872054993724,10cyclictest23-21ksoftirqd/121:55:241
1872054993710,5cyclictest131rcu_preempt19:50:011
1872054993710,5cyclictest131rcu_preempt19:50:001
1872047993730,4cyclictest16950irq/224-800000021:00:010
1872047993730,4cyclictest16950irq/224-800000021:00:000
187206099362,3cyclictest131rcu_preempt21:10:003
1872054993624,10cyclictest23-21ksoftirqd/120:35:011
1872054993615,16cyclictest275-21dbus-daemon20:15:011
187204799362,5cyclictest1-21systemd19:20:010
187206099351,3cyclictest131rcu_preempt20:55:013
187206099351,3cyclictest131rcu_preempt19:40:013
187205599352,3cyclictest131rcu_preempt20:10:002
187205499354,25cyclictest284-21systemd-logind19:45:001
1872060993421,4cyclictest37-21ksoftirqd/319:10:023
187206099341,4cyclictest131rcu_preempt19:25:033
1872054993412,5cyclictest23-21ksoftirqd/120:05:021
1872054993412,5cyclictest23-21ksoftirqd/120:05:021
187205499336,18cyclictest1875906-21munin-plugin-st20:24:591
187205599321,23cyclictest0-21swapper/219:25:012
187205599306,3cyclictest131rcu_preempt19:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional