You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-23 - 06:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat May 23, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19423993231,1cyclictest1553-21lspci21:56:290
19425992828,0cyclictest0-21swapper/221:56:282
1942699250,24cyclictest0-21swapper/323:59:363
1942699250,24cyclictest0-21swapper/321:51:283
1942699250,24cyclictest0-21swapper/300:23:403
19425992523,1cyclictest0-21swapper/221:37:552
1942499250,25cyclictest10200-21ssh21:35:461
1942499250,1cyclictest12754-21diskmemload23:17:511
1942399250,1cyclictest30069-21cut23:17:550
19426992423,0cyclictest0-21swapper/323:23:293
1942699240,2cyclictest1437-21ls19:40:003
19425992424,0cyclictest0-21swapper/222:05:082
19425992422,2cyclictest31566-21sh23:18:532
1942599240,23cyclictest0-21swapper/222:52:342
1942599240,23cyclictest0-21swapper/222:47:332
1942599240,23cyclictest0-21swapper/222:22:452
1942599240,23cyclictest0-21swapper/221:24:042
1942599240,23cyclictest0-21swapper/200:20:012
1942599240,23cyclictest0-21swapper/200:02:492
1942599240,0cyclictest12754-21diskmemload22:33:322
19424992423,1cyclictest3789-21ssh22:26:281
19424992422,1cyclictest19987-21sh21:16:541
1942499240,23cyclictest30494-21apt-get23:47:511
1942499240,23cyclictest12754-21diskmemload00:15:191
1942499240,23cyclictest0-21swapper/123:52:531
1942499240,0cyclictest0-21swapper/123:06:071
19423992423,1cyclictest13808-21ssh21:38:140
1942399240,24cyclictest0-21swapper/023:48:400
1942399240,24cyclictest0-21swapper/022:46:330
1942399240,24cyclictest0-21swapper/021:15:510
1942399240,0cyclictest12754-21diskmemload22:20:540
19426992323,0cyclictest12754-21diskmemload22:28:543
19426992323,0cyclictest12754-21diskmemload21:30:543
19426992323,0cyclictest0-21swapper/323:13:143
19426992323,0cyclictest0-21swapper/323:13:143
19426992323,0cyclictest0-21swapper/323:02:283
19426992323,0cyclictest0-21swapper/322:23:573
19426992323,0cyclictest0-21swapper/322:06:453
19426992322,1cyclictest12754-21diskmemload22:19:223
19426992321,1cyclictest11956-21apt-get22:05:003
1942699230,2cyclictest2072-21sh23:50:023
1942699230,23cyclictest12754-21diskmemload23:18:293
1942699230,23cyclictest0-21swapper/323:43:273
1942699230,23cyclictest0-21swapper/322:47:193
1942699230,23cyclictest0-21swapper/322:35:033
1942699230,22cyclictest19654-21fschecks_count23:37:513
1942699230,22cyclictest1886-21runrttasks00:18:073
1942699230,22cyclictest13502-21rm00:00:073
1942699230,22cyclictest11086-21dump-pmu-power20:00:013
1942699230,0cyclictest0-21swapper/321:15:493
1942699230,0cyclictest0-21swapper/300:09:103
19425992323,0cyclictest0-21swapper/223:50:502
19425992323,0cyclictest0-21swapper/200:31:252
19425992323,0cyclictest0-21swapper/200:06:292
19425992322,1cyclictest12754-21diskmemload22:10:152
19425992322,0cyclictest0-21swapper/223:12:132
19425992322,0cyclictest0-21swapper/223:12:132
1942599232,20cyclictest0-21swapper/221:18:052
19425992321,1cyclictest0-21swapper/223:01:552
19425992321,1cyclictest0-21swapper/221:31:422
19425992321,1cyclictest0-21swapper/221:27:252
19425992321,1cyclictest0-21swapper/200:13:232
1942599230,23cyclictest0-21swapper/222:40:402
1942599230,23cyclictest0-21swapper/221:50:112
1942599230,22cyclictest8442-21diskmemload23:56:122
1942599230,22cyclictest18803-21ssh23:37:242
1942599230,22cyclictest14681-21awk00:30:002
1942599230,0cyclictest0-21swapper/223:46:192
1942599230,0cyclictest0-21swapper/222:25:372
19424992323,0cyclictest12754-21diskmemload22:30:581
19424992323,0cyclictest0-21swapper/122:55:041
19424992323,0cyclictest0-21swapper/121:41:181
19424992323,0cyclictest0-21swapper/100:35:011
19424992323,0cyclictest0-21swapper/100:01:141
19424992322,1cyclictest0-21swapper/123:44:411
19424992321,1cyclictest0-21swapper/123:59:361
19424992321,1cyclictest0-21swapper/121:26:421
19424992320,2cyclictest15807-21sh22:37:211
1942499230,2cyclictest2073-21ssh22:24:271
1942499230,23cyclictest0-21swapper/123:30:051
1942499230,23cyclictest0-21swapper/123:00:131
1942499230,23cyclictest0-21swapper/122:48:531
1942499230,23cyclictest0-21swapper/122:40:281
1942499230,23cyclictest0-21swapper/100:22:411
1942499230,22cyclictest24416-21ssh22:15:491
1942499230,22cyclictest19969-21ssh00:34:391
1942499230,22cyclictest19337-21ssh00:05:201
1942499230,22cyclictest0-21swapper/122:50:301
1942499230,22cyclictest0-21swapper/121:20:571
1942499230,1cyclictest686-21apt20:45:001
1942499230,0cyclictest0-21swapper/123:10:071
1942499230,0cyclictest0-21swapper/123:10:071
1942499230,0cyclictest0-21swapper/122:10:351
19423992323,0cyclictest12754-21diskmemload23:00:330
19423992323,0cyclictest0-21swapper/023:40:390
19423992323,0cyclictest0-21swapper/000:16:330
19423992323,0cyclictest0-21swapper/000:13:320
19423992321,2cyclictest16054-21kworker/u8:121:45:200
19423992321,1cyclictest20756-21ssh22:41:170
19423992320,2cyclictest15258-21ssh00:30:350
1942399231,21cyclictest9805-21rm00:25:500
1942399231,21cyclictest62550irq/127-eno122:10:040
1942399230,23cyclictest13360-21ssh22:05:590
1942399230,23cyclictest0-21swapper/022:34:430
1942399230,23cyclictest0-21swapper/021:11:330
1942399230,22cyclictest3262-21rm22:25:380
1942399230,22cyclictest25008-21ssh00:38:550
1942399230,22cyclictest21977-21ssh23:10:560
1942399230,22cyclictest21977-21ssh23:10:560
1942399230,22cyclictest21128-21perf20:20:010
1942399230,22cyclictest0-21swapper/023:51:220
1942399230,22cyclictest0-21swapper/023:07:270
1942399230,0cyclictest12754-21diskmemload23:39:050
1942399230,0cyclictest12754-21diskmemload22:38:090
1942399230,0cyclictest12754-21diskmemload21:53:420
1942399230,0cyclictest0-21swapper/023:33:360
1942399230,0cyclictest0-21swapper/020:12:490
1942399230,0cyclictest0-21swapper/000:08:590
19426992222,0cyclictest12754-21diskmemload22:50:293
19426992222,0cyclictest12754-21diskmemload21:20:103
19426992222,0cyclictest0-21swapper/322:12:373
19426992222,0cyclictest0-21swapper/321:45:053
19426992222,0cyclictest0-21swapper/300:27:343
19426992221,1cyclictest0-21swapper/323:46:353
19426992221,1cyclictest0-21swapper/323:32:263
19426992221,1cyclictest0-21swapper/322:55:433
19426992220,1cyclictest20325-21diskmemload22:40:363
19426992220,1cyclictest0-21swapper/321:14:223
19426992220,1cyclictest0-21swapper/300:30:303
1942699220,2cyclictest29048-21sh19:30:013
1942699220,22cyclictest0-21swapper/321:35:283
1942699220,21cyclictest30554-21ssh21:25:183
1942699220,21cyclictest25112-21sh00:10:323
1942699220,21cyclictest0-21swapper/323:25:483
1942699220,21cyclictest0-21swapper/323:05:203
1942699220,21cyclictest0-21swapper/321:55:313
1942699220,21cyclictest0-21swapper/320:52:483
1942699220,21cyclictest0-21swapper/320:15:013
1942699220,21cyclictest0-21swapper/300:35:033
1942699220,1cyclictest606-21dbus-daemon19:35:003
1942699220,1cyclictest30730-21perf20:40:003
1942699220,1cyclictest0-21swapper/322:32:063
1942699220,1cyclictest0-21swapper/321:40:143
19425992222,0cyclictest0-21swapper/222:36:332
19425992220,2cyclictest911-21ssh23:21:012
19425992220,1cyclictest0-21swapper/223:40:342
19425992220,1cyclictest0-21swapper/223:25:012
19425992220,1cyclictest0-21swapper/221:43:382
19425992220,1cyclictest0-21swapper/221:10:112
1942599220,21cyclictest5977-21apt-get20:57:502
1942599220,21cyclictest5383-21ssh22:56:282
1942599220,21cyclictest31201-21apt-get20:42:492
1942599220,21cyclictest12754-21diskmemload22:17:162
1942599220,21cyclictest12754-21diskmemload22:00:242
1942599220,1cyclictest12754-21diskmemload23:31:542
1942599220,1cyclictest0-21swapper/200:35:452
1942599220,1cyclictest0-21swapper/200:16:312
1942599220,0cyclictest0-21swapper/223:05:432
1942599220,0cyclictest0-21swapper/221:45:362
19424992222,0cyclictest0-21swapper/122:07:131
19424992222,0cyclictest0-21swapper/122:00:141
19424992222,0cyclictest0-21swapper/121:50:371
19424992222,0cyclictest0-21swapper/120:27:491
19424992220,2cyclictest5927-21ssh21:59:381
19424992220,1cyclictest10381-21grep21:05:011
19424992219,2cyclictest29056-21cron19:30:011
19424992217,4cyclictest7905-21sendmail21:00:011
1942499220,21cyclictest24278-21sendmail-msp19:20:011
1942499220,21cyclictest21598-21apt-get20:22:491
1942499220,21cyclictest17380-21ssh23:35:251
1942499220,21cyclictest0-21swapper/120:55:011
1942499220,1cyclictest22878-21ssh21:47:371
1942499220,1cyclictest0-21swapper/123:25:051
19423992222,0cyclictest12754-21diskmemload23:56:380
19423992222,0cyclictest0-21swapper/021:30:440
19423992220,1cyclictest32534-21rm21:27:510
1942399220,2cyclictest14727-21ssh00:01:460
1942399220,22cyclictest0-21swapper/022:55:130
1942399220,21cyclictest62550irq/127-eno121:40:500
1942399220,21cyclictest24404-21sh22:15:480
1942399220,21cyclictest12751-21grep21:10:000
1942399220,21cyclictest0-21swapper/000:20:000
1942399220,1cyclictest6646-21rm23:26:120
1942399220,1cyclictest0-21swapper/020:42:490
1942399220,0cyclictest12754-21diskmemload22:00:020
1942399220,0cyclictest0-21swapper/021:20:150
1942699210,20cyclictest0-21swapper/320:22:493
19425992119,1cyclictest19881-21apt-get19:12:492
19425992118,2cyclictest11117-21latency_hist20:00:012
1942599210,20cyclictest0-21swapper/220:32:472
1942599210,0cyclictest0-21swapper/221:02:492
19424992119,1cyclictest0-21swapper/119:42:481
1942499210,21cyclictest0-21swapper/123:20:471
19423992121,0cyclictest5517-21perf20:55:010
19423992120,1cyclictest0-21swapper/020:07:480
19423992120,1cyclictest0-21swapper/019:27:480
19423992118,2cyclictest28322-21apt20:35:000
1942399210,21cyclictest0-21swapper/022:50:200
19426992020,0cyclictest0-21swapper/320:42:483
1942699200,19cyclictest26387-21apt-get20:32:473
1942699200,0cyclictest0-21swapper/321:02:493
1942699200,0cyclictest0-21swapper/320:17:473
19425992020,0cyclictest23992-21apt-get20:27:492
19425992020,0cyclictest0-21swapper/220:17:472
19425992020,0cyclictest0-21swapper/219:32:472
19425992019,1cyclictest0-21swapper/219:42:482
1942599200,18cyclictest313-21systemd-journal19:20:012
19424992020,0cyclictest10838-21apt-get21:07:481
19424992020,0cyclictest0-21swapper/120:10:011
19424992019,1cyclictest31931-21apt-get19:37:491
19424992018,1cyclictest11093-21ls20:00:011
1942499200,0cyclictest0-21swapper/120:12:481
19423992020,0cyclictest0-21swapper/020:27:490
19423992020,0cyclictest0-21swapper/019:32:470
19423992019,1cyclictest1461-21munin-run19:40:000
19423992018,1cyclictest24262-21latency_hist19:20:010
1942399200,0cyclictest0-21swapper/021:02:490
19426991918,0cyclictest0-21swapper/321:07:483
19425991919,0cyclictest0-21swapper/220:02:502
19425991917,1cyclictest15917-21dump-pmu-power20:10:012
19424991919,0cyclictest19170-21apt-get20:17:471
19424991916,2cyclictest30704-21sh20:40:001
19426991818,0cyclictest0-21swapper/319:12:493
19426991816,1cyclictest26663-21idleruntime-cro19:25:013
19426991816,1cyclictest25908-21idleruntime-cro20:30:003
19425991818,0cyclictest0-21swapper/220:37:482
19424991818,0cyclictest6725-21apt-get19:52:491
19424991818,0cyclictest29531-21apt-get19:32:471
19424991818,0cyclictest11591-21apt-get20:02:501
1942399186,4cyclictest4298-21/usr/sbin/munin19:47:490
19423991818,0cyclictest1174-21apt-get20:47:500
19423991818,0cyclictest0-21swapper/020:57:500
19423991818,0cyclictest0-21swapper/020:40:000
19423991818,0cyclictest0-21swapper/019:22:480
19423991818,0cyclictest0-21swapper/019:12:490
19426991717,0cyclictest0-21swapper/319:47:483
19426991715,1cyclictest24280-21munin-plugin-st19:20:013
19425991717,0cyclictest0-21swapper/220:47:502
19425991717,0cyclictest0-21swapper/219:47:482
19425991717,0cyclictest0-21swapper/219:37:492
19423991717,0cyclictest1936-21apt-get19:42:480
19426991616,0cyclictest0-21swapper/319:52:483
19425991615,1cyclictest16379-21apt-get20:12:492
1942599160,1cyclictest24742-21apt-get19:22:482
19424991616,0cyclictest0-21swapper/120:32:471
1942499160,15cyclictest0-21swapper/119:12:491
19423991614,1cyclictest9088-21acpi19:57:480
1942599150,14cyclictest0-21swapper/219:52:542
19424991515,0cyclictest4330-21apt-get19:47:481
1942399154,10cyclictest606-21dbus-daemon19:53:010
19423991513,1cyclictest13515-21cron20:05:010
1942699142,9cyclictest611-21irqbalance19:40:293
19426991414,0cyclictest0-21swapper/320:45:323
1942699140,14cyclictest0-21swapper/320:05:093
1942699140,0cyclictest0-21swapper/320:55:563
1942699140,0cyclictest0-21swapper/320:00:183
19425991414,0cyclictest0-21swapper/221:05:502
1942599140,14cyclictest0-21swapper/220:20:192
1942599140,0cyclictest0-21swapper/220:51:272
1942599140,0cyclictest0-21swapper/219:25:062
1942399140,14cyclictest0-21swapper/020:20:580
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional