You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-16 - 15:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sat May 16, 2026 12:45:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1211364991291,125cyclictest0-21swapper/009:10:030
1211369998113,44cyclictest1282766-21rm11:25:031
1211364997326,12cyclictest1303222-21ssh12:07:390
1211369997237,31cyclictest1220014-21munin-plugin-st09:15:001
121136999722,12cyclictest131rcu_preempt11:55:021
1211373997142,8cyclictest1202348-21kworker/2:009:20:382
1211373997142,8cyclictest1202348-21kworker/2:009:20:382
1211373997042,8cyclictest1260930-21kworker/2:111:06:512
1211369996731,8cyclictest1212331-21sshd07:28:481
1211373996642,9cyclictest1260930-21kworker/2:111:25:592
1211364996625,12cyclictest12-21ksoftirqd/010:55:310
1211373996537,9cyclictest1260930-21kworker/2:111:54:152
1211364996519,21cyclictest12-21ksoftirqd/010:35:020
1211364996519,21cyclictest12-21ksoftirqd/010:35:020
121136499641,31cyclictest0-21swapper/010:23:080
1211373996339,7cyclictest1202348-21kworker/2:009:55:552
1211373996338,10cyclictest1234502-21kworker/2:210:03:182
1211369996330,9cyclictest23-21ksoftirqd/112:27:251
121136499631,43cyclictest0-21swapper/010:02:460
1211364996217,22cyclictest1234971-21ssh09:45:360
1211364996216,21cyclictest1238532-21ssh09:53:160
1211373996051,6cyclictest1234502-21kworker/2:210:25:242
1211364996031,12cyclictest1240557-21ssh09:57:230
1211364996025,19cyclictest1234499-21ssh09:45:010
1211364995930,13cyclictest1224240-21ssh09:23:330
1211364995930,13cyclictest1224240-21ssh09:23:330
1211364995915,27cyclictest16850irq/134-800000010:29:130
121136499591,48cyclictest1272954-21ssh11:05:020
1211364995830,25cyclictest1288915-21ssh11:37:590
1211364995829,13cyclictest1275204-21ssh11:09:370
1211364995829,13cyclictest1226249-21ssh09:27:430
1211364995829,13cyclictest1226249-21ssh09:27:430
1211364995828,15cyclictest1313118-21ssh12:28:070
1211364995824,24cyclictest1260398-21ssh10:38:520
121136499581,42cyclictest1309521-21ssh12:20:360
1211364995623,29cyclictest1297146-21date11:55:030
1211364995622,18cyclictest1316348-21ssh12:35:010
1211364995622,18cyclictest1316348-21ssh12:35:000
1211373995547,5cyclictest1234502-21kworker/2:210:36:072
1211373995546,6cyclictest1260930-21kworker/2:111:48:282
1211373995546,5cyclictest1260930-21kworker/2:112:20:012
121136499553,34cyclictest131rcu_preempt12:14:470
121136499552,43cyclictest1290066-21kworker/u8:411:49:460
1211364995513,38cyclictest12-21ksoftirqd/011:17:470
1211373995446,5cyclictest1260930-21kworker/2:112:38:412
1211373995446,5cyclictest1260930-21kworker/2:111:32:532
1211373995444,6cyclictest1234502-21kworker/2:210:32:102
1211373995444,6cyclictest1234502-21kworker/2:210:32:092
121137399544,25cyclictest1211800-21kworker/u8:108:25:032
1211373995423,8cyclictest1260930-21kworker/2:111:17:312
1211364995423,15cyclictest1282601-21ssh11:25:010
1211378995335,15cyclictest3467602-21systemd-journal11:30:003
1211378995335,13cyclictest3467602-21systemd-journal12:05:013
1211378995335,13cyclictest3467602-21systemd-journal12:05:013
1211373995345,5cyclictest1285542-21kworker/2:012:10:262
1211373995345,5cyclictest1234502-21kworker/2:210:12:462
1211373995345,5cyclictest1202348-21kworker/2:007:15:002
1211373995344,5cyclictest1234502-21kworker/2:210:56:532
1211373995336,7cyclictest1234502-21kworker/2:211:03:132
121136999531,29cyclictest1214930-21kworker/u8:408:40:021
121136499532,6cyclictest0-21swapper/007:20:000
1211364995326,12cyclictest1220326-21ssh09:15:200
1211364995324,14cyclictest1248289-21ssh10:13:460
1211364995322,19cyclictest12-21ksoftirqd/010:05:590
1211364995319,16cyclictest0-21swapper/010:15:440
121136499531,44cyclictest0-21swapper/011:44:550
121136499531,43cyclictest0-21swapper/011:12:000
1211373995244,5cyclictest1260930-21kworker/2:112:02:252
1211373995244,5cyclictest1260930-21kworker/2:112:02:242
1211373995244,5cyclictest1260930-21kworker/2:111:20:252
1211373995243,6cyclictest1260930-21kworker/2:112:31:372
1211373995243,6cyclictest1260930-21kworker/2:112:31:372
1211373995243,6cyclictest1234502-21kworker/2:210:08:222
1211373995243,6cyclictest1202348-21kworker/2:009:35:442
1211373995242,7cyclictest1234502-21kworker/2:210:23:142
1211364995226,12cyclictest1230408-21ssh09:36:130
1211364995225,12cyclictest1318222-21ssh12:38:480
1211364995218,23cyclictest12-21ksoftirqd/010:45:010
1211378995136,10cyclictest37-21ksoftirqd/310:15:003
1211378995117,24cyclictest37-21ksoftirqd/310:39:033
1211378995115,9cyclictest37-21ksoftirqd/309:20:003
1211373995144,4cyclictest1202348-21kworker/2:009:54:542
1211373995143,5cyclictest1260930-21kworker/2:112:22:322
1211373995143,5cyclictest1260930-21kworker/2:112:00:022
1211373995143,5cyclictest1260930-21kworker/2:111:13:382
1211373995143,5cyclictest1202348-21kworker/2:009:47:392
1211373995143,5cyclictest1202348-21kworker/2:009:33:462
1211373995143,5cyclictest1202348-21kworker/2:009:28:462
1211373995143,5cyclictest1202348-21kworker/2:009:28:452
1211373995142,5cyclictest1285542-21kworker/2:012:06:142
1211373995142,5cyclictest1202348-21kworker/2:008:15:022
1211378995036,10cyclictest1-21systemd08:15:013
1211373995043,4cyclictest1202348-21kworker/2:009:15:472
1211373995042,5cyclictest1260930-21kworker/2:111:40:402
1211373995042,5cyclictest1260930-21kworker/2:111:36:212
1211373995042,5cyclictest1202348-21kworker/2:009:42:242
1211373995042,5cyclictest1202348-21kworker/2:008:50:022
1211373995041,5cyclictest1260930-21kworker/2:112:27:032
1211373995041,5cyclictest1202348-21kworker/2:007:24:592
1211373995040,5cyclictest1202348-21kworker/2:007:45:002
1211373995040,5cyclictest1202348-21kworker/2:007:45:002
1211373995010,10cyclictest131rcu_preempt10:45:022
1211364995024,11cyclictest1228925-21ssh09:33:180
1211364995023,11cyclictest12-21ksoftirqd/012:00:010
1211364995012,30cyclictest16850irq/134-800000011:26:340
1211378994934,11cyclictest275-21dbus-daemon11:45:013
1211378994926,17cyclictest275-21dbus-daemon09:40:013
1211373994941,5cyclictest1234502-21kworker/2:210:18:042
1211373994941,5cyclictest1216281-21kworker/2:209:10:412
1211364994932,3cyclictest16850irq/134-800000010:53:200
1211364994925,20cyclictest1300335-21ssh12:01:320
1211364994925,20cyclictest1300335-21ssh12:01:310
1211364994922,12cyclictest1306914-21ssh12:15:140
121137899485,8cyclictest37-21ksoftirqd/311:55:023
1211378994831,14cyclictest37-21ksoftirqd/310:20:053
1211373994840,5cyclictest1234502-21kworker/2:210:51:112
1211373994831,5cyclictest30-21ksoftirqd/207:50:002
1211364994828,16cyclictest1265568-21ssh10:49:410
121137899473,6cyclictest131rcu_preempt09:25:273
121137899473,6cyclictest131rcu_preempt09:25:273
1211378994732,12cyclictest1-21systemd12:25:003
1211378994728,14cyclictest3467602-21systemd-journal10:35:003
1211378994728,14cyclictest3467602-21systemd-journal10:35:003
121137899471,6cyclictest131rcu_preempt09:50:473
1211373994739,5cyclictest1234502-21kworker/2:210:48:012
1211369994732,11cyclictest275-21dbus-daemon08:20:011
121136999471,3cyclictest131rcu_preempt10:12:471
121137899468,30cyclictest1212850-21modprobe07:40:003
1211378994630,12cyclictest3467602-21systemd-journal09:35:013
1211378994614,24cyclictest1211550-21modprobe07:15:003
1211373994639,4cyclictest1202348-21kworker/2:009:00:492
1211373994638,5cyclictest1202348-21kworker/2:009:10:012
1211369994628,12cyclictest1318824-21latency_hist12:40:011
121136499468,30cyclictest16850irq/134-800000011:35:000
121136499464,38cyclictest0-21swapper/008:40:000
1211364994628,13cyclictest3467602-21systemd-journal08:20:010
1211378994528,13cyclictest3467602-21systemd-journal11:49:593
1211378994525,15cyclictest275-21dbus-daemon09:10:013
1211373994529,4cyclictest30-21ksoftirqd/208:00:012
121137899444,5cyclictest131rcu_preempt12:10:013
1211378994428,12cyclictest3467602-21systemd-journal10:00:003
1211378994417,20cyclictest37-21ksoftirqd/310:50:183
1211378994415,20cyclictest37-21ksoftirqd/311:35:003
1211378994413,5cyclictest131rcu_preempt12:00:023
1211373994427,13cyclictest1-21systemd07:35:012
121136999443,4cyclictest131rcu_preempt08:15:021
1211369994430,10cyclictest275-21dbus-daemon09:05:011
1211369994429,10cyclictest275-21dbus-daemon10:10:011
1211369994429,10cyclictest1-21systemd09:55:001
1211369994427,13cyclictest1-21systemd11:05:001
121136499447,28cyclictest1211367-21grep07:10:020
121136499447,28cyclictest1211367-21grep07:10:020
1211378994329,11cyclictest37-21ksoftirqd/311:00:013
1211378994325,13cyclictest1-21systemd11:39:593
121137899431,5cyclictest131rcu_preempt10:26:293
1211378994311,6cyclictest131rcu_preempt11:20:513
1211373994336,4cyclictest1202348-21kworker/2:007:28:532
1211373994327,12cyclictest1-21systemd08:45:012
121136999438,8cyclictest23-21ksoftirqd/107:35:021
1211369994333,7cyclictest1287373-21sh11:35:001
1211369994331,8cyclictest1-21systemd08:30:011
121136999432,4cyclictest131rcu_preempt10:00:021
121136999432,4cyclictest131rcu_preempt09:42:281
121136999432,3cyclictest131rcu_preempt10:22:571
1211369994321,15cyclictest275-21dbus-daemon10:30:001
1211364994331,9cyclictest275-21dbus-daemon07:35:010
121137899429,6cyclictest131rcu_preempt12:20:033
1211378994230,9cyclictest37-21ksoftirqd/310:01:533
1211378994230,9cyclictest37-21ksoftirqd/308:50:023
1211378994228,10cyclictest37-21ksoftirqd/312:34:433
1211378994228,10cyclictest37-21ksoftirqd/312:34:423
1211378994224,13cyclictest275-21dbus-daemon09:05:003
1211378994217,5cyclictest131rcu_preempt09:48:163
1211378994210,6cyclictest131rcu_preempt08:25:023
1211373994235,4cyclictest1202348-21kworker/2:007:35:582
1211373994228,11cyclictest3467602-21systemd-journal08:05:022
1211369994230,9cyclictest23-21ksoftirqd/108:50:021
1211369994229,10cyclictest23-21ksoftirqd/111:59:581
1211369994228,11cyclictest23-21ksoftirqd/108:10:021
1211369994226,12cyclictest23-21ksoftirqd/107:25:011
121136999422,4cyclictest131rcu_preempt11:00:001
1211369994214,6cyclictest131rcu_preempt10:34:301
1211369994214,6cyclictest131rcu_preempt10:34:291
1211369994210,3cyclictest131rcu_preempt12:32:221
1211369994210,3cyclictest131rcu_preempt12:32:211
1211364994224,14cyclictest3467602-21systemd-journal07:45:010
1211364994224,14cyclictest3467602-21systemd-journal07:45:010
121137899414,26cyclictest1306790-21cat12:15:023
1211378994126,11cyclictest1215995-21tr08:40:003
1211378994123,4cyclictest37-21ksoftirqd/311:04:213
1211378994117,20cyclictest37-21ksoftirqd/310:43:593
1211373994132,5cyclictest1202348-21kworker/2:008:35:012
1211373994132,5cyclictest1202348-21kworker/2:008:35:012
1211369994127,10cyclictest23-21ksoftirqd/109:46:191
1211369994126,12cyclictest1-21systemd11:10:021
1211369994126,11cyclictest1224983-21latency_hist09:25:021
1211369994126,11cyclictest1224983-21latency_hist09:25:021
121136999411,5cyclictest0-21swapper/110:48:581
1211369994111,8cyclictest131rcu_preempt12:21:011
1211364994135,3cyclictest16950irq/224-800000008:05:010
1211364994127,7cyclictest12-21ksoftirqd/008:35:000
1211364994127,7cyclictest12-21ksoftirqd/008:35:000
1211364994116,21cyclictest1212146-21latency_hist07:25:010
121136499411,36cyclictest1212898-21unin-run07:40:010
121137899407,6cyclictest131rcu_preempt11:10:283
1211378994032,5cyclictest1217540-21diskmemload11:20:033
1211378994026,8cyclictest1251446-21latency_hist10:20:023
1211378994026,10cyclictest37-21ksoftirqd/311:08:243
1211378994024,12cyclictest284-21systemd-logind07:23:593
1211378994022,12cyclictest309-21in:imuxsock08:00:013
1211378994022,11cyclictest309-21in:imuxsock07:25:473
1211378994011,8cyclictest131rcu_preempt09:12:013
1211378994010,6cyclictest131rcu_preempt09:43:383
1211373994031,5cyclictest1202348-21kworker/2:008:36:472
121136999405,8cyclictest23-21ksoftirqd/112:09:581
1211369994027,9cyclictest23-21ksoftirqd/110:44:451
1211369994025,11cyclictest364-21snmpd11:45:031
1211369994024,11cyclictest3467602-21systemd-journal09:35:021
121136999402,3cyclictest131rcu_preempt09:15:271
121136999401,35cyclictest1217746-21wc09:10:021
1211364994026,10cyclictest12-21ksoftirqd/008:50:020
1211364994025,10cyclictest3467602-21systemd-journal09:00:000
1211364994024,12cyclictest1213521-21sed07:50:010
121136499402,3cyclictest131rcu_preempt08:55:000
121136499402,33cyclictest1211740-21ls07:15:020
121136499402,11cyclictest1217546-21munin-run09:10:000
121137899398,6cyclictest131rcu_preempt08:20:003
121137899393,8cyclictest37-21ksoftirqd/308:35:013
121137899393,8cyclictest37-21ksoftirqd/308:35:013
1211378993927,10cyclictest37-21ksoftirqd/310:08:463
121137899391,5cyclictest0-21swapper/308:30:023
1211378993910,7cyclictest1222730-21ssh09:20:093
1211378993910,7cyclictest1222730-21ssh09:20:083
1211378993910,19cyclictest1211818-21modprobe07:20:003
1211373993931,5cyclictest1202348-21kworker/2:008:55:332
1211373993930,6cyclictest1202348-21kworker/2:007:52:442
1211373993930,5cyclictest1202348-21kworker/2:007:20:012
1211369993931,5cyclictest23-21ksoftirqd/111:27:481
1211369993926,9cyclictest1-21systemd07:55:011
1211369993926,10cyclictest1280202-21systemd-run11:20:011
1211369993925,10cyclictest1251304-21run-parts10:20:011
121136499391,32cyclictest1214403-21sshd08:08:470
1211378993828,5cyclictest1265737-21latency_hist10:50:013
1211378993825,11cyclictest37-21ksoftirqd/308:00:033
1211378993824,11cyclictest37-21ksoftirqd/312:37:533
1211378993824,11cyclictest37-21ksoftirqd/312:29:013
121137899381,5cyclictest131rcu_preempt09:00:033
1211373993832,3cyclictest1202348-21kworker/2:008:54:572
121136999383,5cyclictest131rcu_preempt11:36:081
1211369993827,9cyclictest23-21ksoftirqd/111:14:371
1211369993826,9cyclictest23-21ksoftirqd/108:53:471
1211369993824,10cyclictest23-21ksoftirqd/109:31:091
121136999382,3cyclictest131rcu_preempt10:37:501
121136999381,34cyclictest275-21dbus-daemon12:20:011
1211369993812,5cyclictest131rcu_preempt10:00:111
1211364993825,10cyclictest12-21ksoftirqd/009:01:470
121136499382,32cyclictest3467602-21systemd-journal08:27:470
1211364993820,14cyclictest1212339-21sshd07:29:470
121137899377,7cyclictest131rcu_preempt08:45:003
121137899371,5cyclictest131rcu_preempt07:55:013
1211378993715,15cyclictest3467602-21systemd-journal07:50:003
1211373993729,5cyclictest1202348-21kworker/2:008:27:102
1211373993725,9cyclictest1214343-21date08:05:022
121136999373,5cyclictest131rcu_preempt12:02:291
121136999373,5cyclictest131rcu_preempt12:02:291
1211369993727,8cyclictest23-21ksoftirqd/112:10:181
1211369993725,9cyclictest23-21ksoftirqd/111:46:021
1211369993712,15cyclictest23-21ksoftirqd/109:27:071
1211369993712,15cyclictest23-21ksoftirqd/109:27:071
1211364993729,6cyclictest19350irq/34-mmc108:10:060
1211364993724,10cyclictest3467602-21systemd-journal07:59:020
121137899364,6cyclictest131rcu_preempt07:35:003
1211378993624,9cyclictest37-21ksoftirqd/308:10:013
1211378993612,17cyclictest3467602-21systemd-journal07:45:013
1211378993612,17cyclictest3467602-21systemd-journal07:45:013
121137399361,7cyclictest3467602-21systemd-journal08:16:472
1211369993626,8cyclictest23-21ksoftirqd/110:50:171
1211369993622,11cyclictest3467602-21systemd-journal08:32:031
1211369993622,11cyclictest3467602-21systemd-journal08:32:021
1211364993624,8cyclictest12-21ksoftirqd/007:55:020
1211369993527,5cyclictest1213441-21latency_hist07:50:011
1211369993523,9cyclictest23-21ksoftirqd/108:45:011
1211378993421,10cyclictest37-21ksoftirqd/308:51:473
121136999343,6cyclictest131rcu_preempt08:56:471
1211369993424,8cyclictest23-21ksoftirqd/107:45:021
1211369993424,8cyclictest23-21ksoftirqd/107:45:021
1211369993419,11cyclictest3467602-21systemd-journal08:25:011
121136999335,8cyclictest23-21ksoftirqd/107:39:541
1211369993326,4cyclictest1211964-21latency_hist07:20:021
1211369993314,14cyclictest275-21dbus-daemon08:00:011
121136999331,3cyclictest131rcu_preempt08:05:001
1211364993311,13cyclictest131rcu_preempt08:22:290
1211373993122,5cyclictest1211472-21latency_hist07:10:022
1211373993122,5cyclictest1211472-21latency_hist07:10:022
121136999314,3cyclictest131rcu_preempt07:10:021
121136999314,3cyclictest131rcu_preempt07:10:021
1211369993118,9cyclictest1211741-21sort07:15:021
121136499311,28cyclictest0-21swapper/008:41:270
121137899292,4cyclictest131rcu_preempt07:10:033
121137899292,4cyclictest131rcu_preempt07:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional