You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-06 - 19:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Mon Apr 06, 2026 12:45:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1245739998828,12cyclictest1315284-21ssh11:36:360
1245739998434,5cyclictest16850irq/134-800000011:45:000
1245739998428,15cyclictest1296222-21ssh10:52:270
1245739998428,15cyclictest1296222-21ssh10:52:260
1245739998427,12cyclictest1269993-21ssh09:51:260
1245739998028,12cyclictest1276331-21ssh10:06:030
1245739998025,14cyclictest1324489-21ssh11:57:590
124573999742,8cyclictest1-21systemd11:00:010
1245739997326,13cyclictest1289205-21ssh10:35:550
1245747997125,41cyclictest1332385-21sh12:16:061
1245747996927,38cyclictest1261356-21sh09:31:421
1245739996933,27cyclictest16950irq/224-800000012:05:020
124573999692,42cyclictest1311875-21ssh11:28:450
1245739996924,11cyclictest1331781-21ssh12:15:010
1245747996729,34cyclictest1289992-21sh10:37:581
1245747996725,36cyclictest1308651-21sh11:20:591
1245747996725,36cyclictest1308651-21sh11:20:591
1245739996719,12cyclictest1322268-21ssh11:52:470
1245747996626,36cyclictest1326625-21sh12:02:571
124574799662,21cyclictest1300596-21ssh11:02:291
124574799662,21cyclictest1300596-21ssh11:02:291
124575199659,40cyclictest1310396-21ssh11:25:032
124575199659,40cyclictest1310396-21ssh11:25:022
1245747996526,35cyclictest1330835-21sh12:12:421
1245747996524,37cyclictest1268892-21sh09:49:161
1245739996524,15cyclictest1283564-21rm10:22:580
1245739996427,14cyclictest1305512-21ssh11:14:050
1245747996324,35cyclictest23-21ksoftirqd/109:36:551
124574799632,17cyclictest1251943-21systemd-run09:10:011
124574799632,14cyclictest1324616-21ssh11:58:181
1245747996321,38cyclictest23-21ksoftirqd/109:12:081
124574799631,9cyclictest1340207-21ssh12:34:311
1245739996328,12cyclictest1261383-21ssh09:31:460
1245739996325,14cyclictest1252975-21rm09:12:150
124574799622,9cyclictest1306889-21ssh11:17:051
124574799621,9cyclictest1295486-21ssh10:50:321
124574799621,9cyclictest1295486-21ssh10:50:321
124574799621,7cyclictest1329301-21ssh12:09:161
124574799621,7cyclictest1255517-21ssh09:18:121
124574799621,10cyclictest1303489-21ssh11:09:281
1245739996227,14cyclictest1285368-21ssh10:27:050
124574799612,9cyclictest1305540-21ssh11:14:181
124574799612,7cyclictest1298923-21ssh10:58:471
124574799612,20cyclictest1270510-21ssh09:52:501
124574799612,14cyclictest1258888-21sshd09:25:421
124574799611,9cyclictest1336611-21ssh12:25:501
124574799611,8cyclictest1313178-21ssh11:31:381
124574799611,8cyclictest1266524-21ssh09:43:421
124574799611,12cyclictest1278742-21ssh10:11:461
124574799611,10cyclictest1316109-21ssh11:38:441
124574799611,10cyclictest1282880-21ssh10:21:111
124573999616,36cyclictest1268482-21kworker/u8:110:30:030
1245739996129,10cyclictest1263324-21ssh09:36:090
1245747996023,33cyclictest23-21ksoftirqd/110:15:351
124574799601,8cyclictest1320449-21ssh11:48:381
124574799601,8cyclictest1284692-21ssh10:25:241
124574799601,8cyclictest1271896-21ssh09:55:511
124574799601,8cyclictest1256663-21ssh09:20:391
124574799601,8cyclictest1256663-21ssh09:20:381
124574799601,13cyclictest364-21snmpd11:42:231
124573999607,31cyclictest151rcuc/007:25:020
124575199593,8cyclictest131rcu_preempt08:19:452
124574799591,8cyclictest1341961-21ssh12:38:381
124574799591,8cyclictest1335778-21ssh12:24:141
124574799591,8cyclictest1294244-21ssh10:47:481
124574799591,8cyclictest1248761-21sshd08:05:091
124574799591,13cyclictest1251711-21cat09:05:021
124574799591,10cyclictest1291343-21ssh10:40:501
1245739995935,19cyclictest16850irq/134-800000010:43:580
124574799582,14cyclictest1277733-21ssh10:09:441
124574799582,12cyclictest1249321-21munin-run08:20:001
124574799582,10cyclictest364-21snmpd08:47:511
124574799581,9cyclictest3467602-21systemd-journal10:34:271
124574799572,6cyclictest1247524-21munin-run07:45:021
124574799571,7cyclictest3467602-21systemd-journal11:54:471
124574799571,16cyclictest1273963-21ssh10:00:291
1245739995720,32cyclictest1252110-21latency_hist09:10:030
1245751995647,6cyclictest1312352-21kworker/2:112:39:282
1245747995631,19cyclictest309-21in:imuxsock08:35:001
124574799562,7cyclictest1248016-21munin-run07:55:001
124574799561,9cyclictest1249821-21sshd08:28:091
124574799561,6cyclictest1249607-21munin-run08:25:011
124574799561,13cyclictest1311298-21ssh11:27:211
124574799551,9cyclictest1246699-21sshd07:26:091
1245756995440,10cyclictest1-21systemd11:10:013
1245751995435,14cyclictest3467602-21systemd-journal12:10:032
124574799547,40cyclictest1246193-21modprobe07:20:001
124574799543,5cyclictest364-21snmpd08:37:491
124574799543,5cyclictest364-21snmpd08:37:481
1245747995428,23cyclictest1245935-21latency_hist07:15:031
124574799542,6cyclictest1249052-21munin-run08:15:011
124574799541,15cyclictest364-21snmpd08:58:271
1245739995427,12cyclictest1332974-21ssh12:17:290
1245739995413,15cyclictest12-21ksoftirqd/010:20:020
1245756995336,12cyclictest3467602-21systemd-journal08:40:013
1245756995336,12cyclictest3467602-21systemd-journal08:40:003
1245751995345,5cyclictest1236510-21kworker/2:210:01:482
1245751995344,6cyclictest1236510-21kworker/2:209:37:522
1245751995344,5cyclictest1251689-21kworker/2:311:56:472
1245751995344,5cyclictest1251689-21kworker/2:310:22:492
1245751995344,5cyclictest1236510-21kworker/2:209:13:012
1245751995341,9cyclictest30-21ksoftirqd/211:00:012
1245751995335,13cyclictest1-21systemd12:10:012
124574799532,16cyclictest1251146-21munin-run08:55:001
124575199528,5cyclictest131rcu_preempt11:47:172
1245751995244,5cyclictest1312352-21kworker/2:111:41:562
1245751995243,6cyclictest1251689-21kworker/2:309:59:002
1245751995243,6cyclictest1236510-21kworker/2:210:26:182
1245751995243,5cyclictest1236510-21kworker/2:210:48:152
1245751995243,5cyclictest1236510-21kworker/2:210:43:322
1245751995243,5cyclictest1236510-21kworker/2:210:10:482
124574799521,10cyclictest364-21snmpd08:02:431
124574799521,10cyclictest364-21snmpd08:02:421
1245739995245,4cyclictest16950irq/224-800000011:20:000
1245739995245,3cyclictest0-21swapper/009:25:020
1245739995245,3cyclictest0-21swapper/009:25:020
1245756995134,13cyclictest275-21dbus-daemon10:30:013
1245756995131,14cyclictest275-21dbus-daemon11:40:013
1245751995143,5cyclictest1251689-21kworker/2:311:35:412
1245751995142,6cyclictest30-21ksoftirqd/210:05:032
1245751995142,6cyclictest1251689-21kworker/2:312:17:112
1245751995142,6cyclictest1236510-21kworker/2:210:35:532
1245751995142,6cyclictest1236510-21kworker/2:209:51:052
1245751995142,5cyclictest1236510-21kworker/2:209:30:082
1245751995140,7cyclictest1236510-21kworker/2:211:07:042
1245751995134,12cyclictest275-21dbus-daemon08:15:012
124574799511,13cyclictest364-21snmpd07:31:061
124574799511,11cyclictest364-21snmpd08:42:491
124573999511,14cyclictest1250980-21cat08:50:010
124575699503,3cyclictest131rcu_preempt11:40:033
1245756995030,13cyclictest275-21dbus-daemon09:10:013
1245751995042,5cyclictest1312352-21kworker/2:112:30:142
1245751995042,5cyclictest1251689-21kworker/2:312:03:272
1245751995042,5cyclictest1251689-21kworker/2:311:51:582
1245751995042,5cyclictest1251689-21kworker/2:311:30:482
1245751995042,5cyclictest1251689-21kworker/2:311:13:192
1245751995042,5cyclictest1251689-21kworker/2:309:49:222
1245751995042,5cyclictest1251689-21kworker/2:309:20:392
1245751995042,5cyclictest1251689-21kworker/2:309:20:382
1245751995042,5cyclictest1249605-21kworker/2:008:55:312
1245751995041,6cyclictest1251689-21kworker/2:312:24:362
1245751995041,6cyclictest1236510-21kworker/2:209:25:232
1245751995041,5cyclictest1251689-21kworker/2:310:18:012
1245751995040,6cyclictest1312352-21kworker/2:112:26:542
1245751995039,7cyclictest30-21ksoftirqd/207:30:002
124574799508,37cyclictest3467602-21systemd-journal07:50:011
124574799508,37cyclictest3467602-21systemd-journal07:50:001
1245739995036,10cyclictest275-21dbus-daemon10:00:010
1245739995032,15cyclictest16950irq/224-800000009:30:020
124575699491,4cyclictest131rcu_preempt11:45:353
1245751994942,5cyclictest1251689-21kworker/2:311:27:562
1245751994942,4cyclictest1251689-21kworker/2:311:15:232
1245751994941,5cyclictest1251689-21kworker/2:311:00:152
1245751994941,5cyclictest1251689-21kworker/2:311:00:142
1245751994941,5cyclictest1251689-21kworker/2:310:32:052
1245751994941,5cyclictest1251689-21kworker/2:309:42:152
1245751994941,5cyclictest1251689-21kworker/2:309:17:082
1245751994940,6cyclictest1236510-21kworker/2:210:51:072
1245751994940,6cyclictest1236510-21kworker/2:210:51:062
124574799492,5cyclictest1247241-21munin-run07:40:001
124574799491,11cyclictest1245752-21cat07:10:011
1245739994930,14cyclictest275-21dbus-daemon09:05:000
1245739994928,7cyclictest1269051-21ssh09:49:390
1245751994840,5cyclictest1249605-21kworker/2:008:52:592
1245751994839,6cyclictest1238383-21kworker/2:107:38:072
1245751994839,6cyclictest1236510-21kworker/2:208:03:102
1245751994839,6cyclictest1236510-21kworker/2:208:03:092
1245739994832,11cyclictest3467602-21systemd-journal11:25:010
1245739994832,11cyclictest3467602-21systemd-journal11:25:000
1245739994827,6cyclictest1300409-21ssh11:02:030
1245739994827,6cyclictest1300409-21ssh11:02:030
1245739994819,25cyclictest1312630-21sh11:30:110
1245751994739,5cyclictest1249605-21kworker/2:009:04:352
1245751994739,5cyclictest1249605-21kworker/2:008:42:432
1245751994739,5cyclictest1249605-21kworker/2:008:35:122
1245751994739,5cyclictest1249605-21kworker/2:008:35:112
1245751994739,5cyclictest1238383-21kworker/2:107:51:092
1245751994739,5cyclictest1238383-21kworker/2:107:16:442
1245751994739,5cyclictest1236510-21kworker/2:209:08:342
1245751994739,5cyclictest1236510-21kworker/2:208:25:132
1245751994739,5cyclictest1236510-21kworker/2:208:23:312
1245751994739,5cyclictest1236510-21kworker/2:208:05:332
1245751994738,6cyclictest1238383-21kworker/2:107:56:232
124574799471,10cyclictest3467602-21systemd-journal07:55:351
1245739994739,5cyclictest16850irq/134-800000009:19:590
1245739994733,11cyclictest1278926-21ssh10:12:200
1245739994730,13cyclictest1329629-21ssh12:10:010
1245739994725,16cyclictest1266961-21ssh09:44:530
1245739994718,22cyclictest210-21jbd2/mmcblk1p2-12:35:430
1245756994638,5cyclictest1336398-21sshd12:25:103
1245756994629,13cyclictest1-21systemd11:15:013
1245756994628,14cyclictest37-21ksoftirqd/308:55:003
1245751994639,4cyclictest1249605-21kworker/2:008:48:142
1245751994638,5cyclictest1249605-21kworker/2:008:31:592
1245751994638,5cyclictest1238383-21kworker/2:107:49:202
1245751994638,5cyclictest1238383-21kworker/2:107:49:192
1245751994638,5cyclictest1238383-21kworker/2:107:30:122
1245751994637,6cyclictest1238383-21kworker/2:107:12:122
1245739994628,14cyclictest1336581-21ssh12:25:450
1245739994628,14cyclictest1294572-21ssh10:48:440
1245756994533,7cyclictest309-21in:imuxsock09:30:003
1245756994527,13cyclictest275-21dbus-daemon09:25:023
1245756994527,13cyclictest275-21dbus-daemon09:25:023
1245756994525,14cyclictest3467602-21systemd-journal10:25:003
1245751994538,4cyclictest1238383-21kworker/2:107:20:182
1245751994537,5cyclictest1238383-21kworker/2:107:42:312
124574799452,10cyclictest364-21snmpd07:21:211
124573999453,8cyclictest1250630-21munin-run08:45:000
1245739994538,4cyclictest16850irq/134-800000011:49:090
1245739994531,9cyclictest1247751-21cron07:50:010
1245739994531,9cyclictest1247751-21cron07:50:010
1245739994528,14cyclictest1338813-21ssh12:30:530
124573999452,3cyclictest131rcu_preempt11:05:550
1245739994521,10cyclictest1334746-21ssh12:21:270
1245756994427,12cyclictest1246183-21cron07:20:003
1245739994429,11cyclictest12-21ksoftirqd/007:20:010
1245756994333,7cyclictest3467602-21systemd-journal12:20:013
1245756994329,11cyclictest37-21ksoftirqd/310:16:143
124575699432,3cyclictest131rcu_preempt09:39:323
124575699431,4cyclictest131rcu_preempt12:10:273
1245739994334,5cyclictest0-21swapper/008:35:020
1245739994327,13cyclictest1275348-21ssh10:04:050
1245756994229,9cyclictest309-21in:imuxsock07:40:003
1245756994228,11cyclictest37-21ksoftirqd/309:18:583
1245756994225,12cyclictest275-21dbus-daemon08:05:013
1245756994225,12cyclictest275-21dbus-daemon08:05:003
1245756994224,12cyclictest275-21dbus-daemon10:10:003
124575699421,3cyclictest131rcu_preempt10:30:123
1245739994235,4cyclictest16950irq/224-800000008:40:000
1245739994235,4cyclictest16950irq/224-800000008:40:000
1245739994227,11cyclictest1248250-21sshd07:59:090
1245739994226,12cyclictest1249014-21sshd08:10:090
1245756994127,10cyclictest37-21ksoftirqd/311:32:383
1245756994125,12cyclictest37-21ksoftirqd/309:15:013
124575699412,11cyclictest346771850lldpd08:05:313
1245756994121,15cyclictest3467602-21systemd-journal08:50:003
1245756994113,5cyclictest131rcu_preempt10:01:133
124575699405,31cyclictest1273527-21munin-run10:00:013
1245756994016,17cyclictest275-21dbus-daemon07:45:013
1245739994033,4cyclictest16950irq/224-800000007:35:010
124573999402,33cyclictest0-21swapper/007:40:000
124573999402,21cyclictest19350irq/34-mmc107:45:010
1245739994022,13cyclictest3467602-21systemd-journal08:20:010
124573999401,34cyclictest1248526-21sshd08:03:100
124573999401,34cyclictest1248526-21sshd08:03:090
1245756993928,8cyclictest37-21ksoftirqd/311:04:333
1245756993928,8cyclictest37-21ksoftirqd/311:04:323
1245756993927,9cyclictest37-21ksoftirqd/312:33:433
1245756993927,9cyclictest3467602-21systemd-journal08:30:003
1245756993927,10cyclictest37-21ksoftirqd/312:05:543
1245756993926,10cyclictest37-21ksoftirqd/309:48:583
1245756993926,10cyclictest3467602-21systemd-journal08:55:103
1245756993924,11cyclictest37-21ksoftirqd/311:57:143
1245756993924,11cyclictest37-21ksoftirqd/310:43:313
124575699391,4cyclictest131rcu_preempt11:54:143
124575699391,3cyclictest131rcu_preempt10:54:443
124575699391,3cyclictest131rcu_preempt10:54:443
1245739993932,4cyclictest16950irq/224-800000008:07:090
1245739993920,13cyclictest275-21dbus-daemon07:55:000
1245756993827,8cyclictest37-21ksoftirqd/312:20:043
1245756993825,10cyclictest1262776-21cut09:35:023
1245756993824,11cyclictest37-21ksoftirqd/311:17:173
1245756993823,11cyclictest37-21ksoftirqd/309:40:253
1245756993822,12cyclictest275-21dbus-daemon07:50:013
1245756993822,12cyclictest275-21dbus-daemon07:50:003
1245756993817,6cyclictest131rcu_preempt10:59:123
1245739993830,5cyclictest19350irq/34-mmc108:58:250
1245739993826,9cyclictest12-21ksoftirqd/008:25:020
124575699379,6cyclictest131rcu_preempt09:55:003
1245756993725,9cyclictest37-21ksoftirqd/311:20:393
1245756993725,9cyclictest37-21ksoftirqd/311:20:383
1245756993724,9cyclictest3467602-21systemd-journal07:30:013
1245756993723,10cyclictest37-21ksoftirqd/312:00:243
1245756993719,15cyclictest275-21dbus-daemon10:40:013
1245756993716,15cyclictest275-21dbus-daemon10:50:003
1245756993626,5cyclictest309-21in:imuxsock07:33:073
124573999365,23cyclictest1245735-21cyclictest07:10:020
124575699356,21cyclictest1248261-21modprobe08:00:003
1245756993527,5cyclictest309-21in:imuxsock08:25:013
1245756993526,5cyclictest1249048-21latency_hist08:15:033
1245756993525,7cyclictest1251636-21sshd09:01:103
1245756993525,5cyclictest37-21ksoftirqd/308:45:003
1245756993523,9cyclictest37-21ksoftirqd/310:11:483
124575699351,3cyclictest131rcu_preempt11:27:523
1245739993522,8cyclictest1251120-21cron08:55:000
1245756993427,4cyclictest1249384-21latency_hist08:20:013
1245756993420,10cyclictest3467602-21systemd-journal12:36:083
1245756993416,13cyclictest275-21dbus-daemon08:35:023
1245739993423,9cyclictest12-21ksoftirqd/007:22:170
124573999341,30cyclictest0-21swapper/008:25:500
124575699339,6cyclictest361rcuc/307:55:013
124573999331,2cyclictest0-21swapper/007:15:010
1245756993123,4cyclictest1246461-21sshd07:24:093
1245756993122,6cyclictest1245791-21sort07:10:023
124575699301,3cyclictest131rcu_preempt07:10:093
1245751993017,10cyclictest1245852-21cat07:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional