You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-01 - 15:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Tue Sep 01, 2026 00:45:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
121565199847,54cyclictest1279055-21rm23:10:030
1215651997571,2cyclictest0-21swapper/000:25:020
1215661996832,10cyclictest30-21ksoftirqd/222:15:022
1215654996812,10cyclictest131rcu_preempt21:35:011
121565199676,30cyclictest1302532-40mandb00:00:060
1215661996641,16cyclictest1236166-21kworker/2:022:01:512
121565199662,40cyclictest150-21pr/ttyS222:00:030
121565199656,6cyclictest131rcu_preempt21:39:330
1215661996440,7cyclictest1220318-21kworker/2:121:24:202
1215661996242,8cyclictest1219728-21kworker/2:222:34:042
121565199622,3cyclictest131rcu_preempt23:55:010
121565199621,43cyclictest1281666-21ssh23:15:350
121565199621,43cyclictest1281666-21ssh23:15:340
1215661996113,40cyclictest291rcuc/223:45:022
1215661996113,40cyclictest291rcuc/223:45:022
1215666996013,6cyclictest131rcu_preempt23:46:133
1215666996011,6cyclictest131rcu_preempt21:13:213
1215661996041,7cyclictest1236166-21kworker/2:021:55:012
1215651996031,14cyclictest1289688-21ssh23:32:380
121565199602,40cyclictest131rcu_preempt23:21:020
1215651996019,35cyclictest16850irq/134-800000022:20:000
1215651995946,9cyclictest16950irq/224-800000021:45:010
121565199592,37cyclictest131rcu_preempt00:22:130
1215651995910,13cyclictest3467602-21systemd-journal21:20:000
1215651995910,13cyclictest3467602-21systemd-journal21:20:000
121566699589,6cyclictest131rcu_preempt22:04:053
121566699582,10cyclictest37-21ksoftirqd/323:41:143
121566699582,10cyclictest37-21ksoftirqd/323:41:143
1215666995816,31cyclictest3467602-21systemd-journal20:45:043
1215666995814,25cyclictest0-21swapper/300:25:033
1215661995848,6cyclictest1236166-21kworker/2:023:50:452
1215651995836,16cyclictest3467602-21systemd-journal23:50:000
1215651995828,14cyclictest1238841-21ssh21:45:250
121566699578,34cyclictest1245623-21ssh22:00:013
1215666995722,9cyclictest37-21ksoftirqd/300:05:013
1215666995710,7cyclictest131rcu_preempt21:30:173
1215661995747,6cyclictest1236166-21kworker/2:022:47:062
1215661995735,5cyclictest1220318-21kworker/2:121:28:502
1215661995735,5cyclictest1220318-21kworker/2:121:28:492
1215661995730,7cyclictest1236166-21kworker/2:022:05:442
1215661995711,6cyclictest131rcu_preempt23:40:002
1215651995725,16cyclictest1306834-21ssh00:09:040
1215651995717,35cyclictest16850irq/134-800000000:15:000
121565199571,39cyclictest131rcu_preempt22:37:060
121566699569,5cyclictest131rcu_preempt22:52:533
121566699567,9cyclictest37-21ksoftirqd/322:06:433
121566699565,5cyclictest131rcu_preempt21:10:013
1215661995636,6cyclictest1295687-21kworker/2:100:25:092
121565199568,43cyclictest12-21ksoftirqd/022:31:360
1215651995628,11cyclictest1259450-21ssh22:29:020
121565199562,3cyclictest131rcu_preempt21:10:010
121566699559,6cyclictest131rcu_preempt00:27:073
1215666995535,15cyclictest3467602-21systemd-journal23:20:013
1215666995535,15cyclictest3467602-21systemd-journal23:20:003
1215666995511,6cyclictest131rcu_preempt22:29:563
1215661995546,6cyclictest1295687-21kworker/2:100:17:312
1215661995545,6cyclictest1236166-21kworker/2:023:32:312
121565199554,3cyclictest131rcu_preempt23:25:140
121565199551,38cyclictest131rcu_preempt23:13:040
121566699548,27cyclictest1216109-21modprobe19:20:003
121566699548,11cyclictest131rcu_preempt00:18:293
121566699547,7cyclictest37-21ksoftirqd/300:00:003
121566699541,8cyclictest3467602-21systemd-journal23:51:083
1215666995413,5cyclictest131rcu_preempt21:36:573
1215666995411,6cyclictest131rcu_preempt19:35:013
1215661995446,5cyclictest1219728-21kworker/2:222:27:222
1215661995410,5cyclictest131rcu_preempt22:42:162
1215651995428,11cyclictest1276013-21ssh23:03:500
1215651995426,13cyclictest1318200-21ssh00:32:480
1215651995418,31cyclictest16850irq/134-800000021:30:030
1215651995418,31cyclictest16850irq/134-800000021:30:020
121565199541,3cyclictest131rcu_preempt20:44:160
121566699538,9cyclictest37-21ksoftirqd/322:39:593
121566699538,6cyclictest131rcu_preempt23:06:213
121566699536,12cyclictest37-21ksoftirqd/300:11:533
121566699534,8cyclictest37-21ksoftirqd/322:44:403
1215666995325,21cyclictest275-21dbus-daemon22:20:013
121566699531,8cyclictest37-21ksoftirqd/321:50:013
1215666995311,5cyclictest131rcu_preempt23:34:063
1215666995310,29cyclictest1217145-21modprobe19:40:003
1215661995346,4cyclictest1295687-21kworker/2:100:03:262
1215661995345,5cyclictest1236166-21kworker/2:023:29:182
1215661995345,5cyclictest1220318-21kworker/2:121:42:022
1215661995345,5cyclictest1219728-21kworker/2:223:15:272
1215661995345,5cyclictest1219728-21kworker/2:223:15:272
1215661995345,5cyclictest1219728-21kworker/2:222:22:572
1215661995342,6cyclictest1219728-21kworker/2:221:33:282
1215661995337,12cyclictest1-21systemd20:00:012
1215661995337,12cyclictest1-21systemd20:00:002
121565199532,36cyclictest1233645-21ssh21:34:520
1215651995322,15cyclictest1304698-21ssh00:04:280
1215651995322,15cyclictest1250374-21ssh22:10:010
1215651995318,31cyclictest16850irq/134-800000000:39:380
121566699529,6cyclictest131rcu_preempt00:39:273
121566699528,11cyclictest37-21ksoftirqd/320:05:023
121566699527,6cyclictest131rcu_preempt22:14:113
121566699526,11cyclictest37-21ksoftirqd/321:20:003
121566699526,11cyclictest37-21ksoftirqd/321:19:593
121566699525,9cyclictest37-21ksoftirqd/323:00:023
121566699523,11cyclictest37-21ksoftirqd/320:25:023
1215666995212,6cyclictest131rcu_preempt22:35:013
1215666995211,5cyclictest131rcu_preempt21:54:383
1215661995244,5cyclictest1295687-21kworker/2:100:38:542
1215661995244,5cyclictest1219728-21kworker/2:223:08:522
1215661995243,6cyclictest1236166-21kworker/2:023:25:002
1215661995243,5cyclictest1295687-21kworker/2:100:13:182
1215661995243,5cyclictest1219728-21kworker/2:223:01:442
121565199525,35cyclictest12-21ksoftirqd/021:20:590
1215651995226,3cyclictest131rcu_preempt21:00:000
1215651995225,12cyclictest1241905-21ssh21:51:540
1215651995218,30cyclictest16850irq/134-800000022:12:010
1215651995218,25cyclictest0-21swapper/019:45:020
1215651995217,31cyclictest16850irq/134-800000022:50:440
1215651995217,31cyclictest16850irq/134-800000022:40:390
1215651995216,32cyclictest16850irq/134-800000023:41:330
1215651995216,32cyclictest16850irq/134-800000023:41:330
1215651995216,31cyclictest16850irq/134-800000022:21:090
1215651995216,31cyclictest16850irq/134-800000021:59:290
121565199521,3cyclictest131rcu_preempt22:49:440
121565199521,3cyclictest131rcu_preempt21:10:510
121566699513,7cyclictest37-21ksoftirqd/323:40:023
121566699512,11cyclictest37-21ksoftirqd/320:00:023
121566699512,11cyclictest37-21ksoftirqd/320:00:013
1215666995114,6cyclictest131rcu_preempt21:22:213
1215666995110,6cyclictest131rcu_preempt22:20:453
1215661995143,5cyclictest1236166-21kworker/2:021:56:182
1215661995143,5cyclictest1220318-21kworker/2:121:46:052
1215661995143,5cyclictest1220318-21kworker/2:121:10:352
1215661995143,5cyclictest1219728-21kworker/2:223:13:162
1215661995143,5cyclictest1219728-21kworker/2:223:00:022
1215661995143,5cyclictest1219728-21kworker/2:222:37:242
1215661995142,5cyclictest1219728-21kworker/2:222:18:392
1215661995141,6cyclictest1220318-21kworker/2:121:16:252
1215661995141,6cyclictest1220318-21kworker/2:121:16:242
1215651995116,31cyclictest16850irq/134-800000000:15:450
121566699509,11cyclictest37-21ksoftirqd/300:35:023
121566699504,6cyclictest131rcu_preempt23:27:283
121566699503,10cyclictest37-21ksoftirqd/323:15:013
121566699501,15cyclictest0-21swapper/323:23:073
121566199506,5cyclictest131rcu_preempt19:40:012
1215661995042,5cyclictest1295687-21kworker/2:100:33:522
1215661995042,5cyclictest1295687-21kworker/2:100:23:172
1215661995042,5cyclictest1295687-21kworker/2:100:09:152
1215661995042,5cyclictest1236166-21kworker/2:023:50:002
1215661995042,5cyclictest1219728-21kworker/2:222:54:582
1215661995041,6cyclictest1207279-21kworker/2:019:12:152
1215661995040,6cyclictest1207279-21kworker/2:020:35:002
1215661995040,5cyclictest1207279-21kworker/2:020:10:002
121565499503,5cyclictest131rcu_preempt00:00:131
1215651995017,29cyclictest16850irq/134-800000022:57:390
121565199501,12cyclictest3467602-21systemd-journal23:39:490
121566699498,8cyclictest37-21ksoftirqd/321:40:183
121566699498,10cyclictest37-21ksoftirqd/300:09:123
121566699496,13cyclictest37-21ksoftirqd/319:30:013
1215666994914,7cyclictest131rcu_preempt22:47:313
1215666994913,5cyclictest131rcu_preempt23:03:403
1215661994941,5cyclictest1219728-21kworker/2:221:36:142
1215661994934,10cyclictest3467602-21systemd-journal20:35:022
1215661994930,15cyclictest0-21swapper/223:59:592
1215661994918,21cyclictest30-21ksoftirqd/219:25:012
121566699487,6cyclictest131rcu_preempt21:27:363
121566699487,6cyclictest131rcu_preempt21:27:353
121566699483,7cyclictest37-21ksoftirqd/320:50:033
121566699483,7cyclictest37-21ksoftirqd/320:50:023
121566699481,6cyclictest0-21swapper/320:15:023
121566699475,7cyclictest37-21ksoftirqd/320:40:023
1215666994736,8cyclictest1217656-21sshd19:47:153
1215661994739,5cyclictest1219728-21kworker/2:221:05:012
1215661994739,5cyclictest1207279-21kworker/2:019:30:002
1215661994730,12cyclictest3467602-21systemd-journal19:55:012
121565499477,4cyclictest1277721-21ssh23:07:251
121565499471,3cyclictest131rcu_preempt21:40:001
1215654994712,5cyclictest131rcu_preempt00:25:031
121565199471,42cyclictest1219271-21unin-run20:20:010
121566699464,9cyclictest37-21ksoftirqd/320:20:023
121566699462,11cyclictest37-21ksoftirqd/321:00:013
121565499464,5cyclictest131rcu_preempt21:25:191
121565499464,5cyclictest131rcu_preempt21:25:181
1215654994624,4cyclictest23-21ksoftirqd/120:00:011
1215654994624,4cyclictest23-21ksoftirqd/120:00:011
1215654994621,6cyclictest23-21ksoftirqd/123:12:221
121566699457,8cyclictest37-21ksoftirqd/320:04:593
121566699457,10cyclictest37-21ksoftirqd/319:55:013
121566699453,7cyclictest37-21ksoftirqd/320:35:003
1215666994516,6cyclictest131rcu_preempt19:10:023
1215661994537,5cyclictest1219728-21kworker/2:220:50:022
1215661994537,5cyclictest1219728-21kworker/2:220:50:022
1215661994537,5cyclictest1207279-21kworker/2:020:02:282
1215661994512,7cyclictest131rcu_preempt20:30:012
121565499457,7cyclictest23-21ksoftirqd/122:51:161
121565499455,5cyclictest131rcu_preempt23:53:161
1215654994528,12cyclictest275-21dbus-daemon23:35:001
121565499451,5cyclictest275-21dbus-daemon20:20:001
121565499451,11cyclictest1302532-40mandb00:00:021
1215651994531,10cyclictest1-21systemd20:30:010
121566699446,6cyclictest131rcu_preempt20:55:003
1215666994435,5cyclictest361rcuc/321:05:013
1215666994411,6cyclictest131rcu_preempt20:30:013
1215661994437,4cyclictest1207279-21kworker/2:020:11:382
1215661994437,4cyclictest1207279-21kworker/2:019:42:412
1215661994436,5cyclictest1219728-21kworker/2:221:07:552
1215661994436,5cyclictest1207279-21kworker/2:019:15:092
1215661994426,3cyclictest30-21ksoftirqd/221:00:002
121565499443,3cyclictest131rcu_preempt21:50:291
1215654994433,8cyclictest1256763-21sshd22:23:151
1215654994432,9cyclictest3467602-21systemd-journal22:10:011
1215654994423,16cyclictest275-21dbus-daemon22:40:011
1215654994422,5cyclictest23-21ksoftirqd/100:34:591
1215654994421,5cyclictest23-21ksoftirqd/122:02:131
1215654994419,6cyclictest23-21ksoftirqd/100:13:241
121565499441,3cyclictest131rcu_preempt23:46:121
1215651994428,12cyclictest275-21dbus-daemon20:35:010
1215651994427,12cyclictest275-21dbus-daemon21:05:020
121566699431,6cyclictest150-21pr/ttyS219:20:203
121566699431,16cyclictest37-21ksoftirqd/319:45:023
1215661994336,4cyclictest1207279-21kworker/2:019:32:212
1215661994315,23cyclictest30-21ksoftirqd/220:25:002
121565499439,6cyclictest131rcu_preempt23:40:151
121565499439,6cyclictest131rcu_preempt23:40:151
121565499437,7cyclictest23-21ksoftirqd/119:42:081
121565499437,6cyclictest23-21ksoftirqd/121:18:441
121565499437,6cyclictest23-21ksoftirqd/121:18:431
121565499435,8cyclictest23-21ksoftirqd/121:56:061
1215654994320,5cyclictest23-21ksoftirqd/121:13:211
1215654994319,5cyclictest23-21ksoftirqd/121:46:271
1215654994319,5cyclictest23-21ksoftirqd/121:44:461
121566699427,6cyclictest131rcu_preempt19:15:013
1215661994234,5cyclictest1219728-21kworker/2:220:43:152
1215661994225,12cyclictest1-21systemd20:20:012
1215654994231,8cyclictest3467602-21systemd-journal22:15:001
121565499422,3cyclictest131rcu_preempt22:28:501
1215654994221,4cyclictest23-21ksoftirqd/122:30:271
1215654994220,6cyclictest23-21ksoftirqd/100:37:531
1215654994220,5cyclictest23-21ksoftirqd/119:24:161
121565499421,8cyclictest364-21snmpd20:26:091
1215654994217,6cyclictest23-21ksoftirqd/100:26:331
121565499421,6cyclictest131rcu_preempt21:07:161
121565199424,3cyclictest131rcu_preempt20:00:020
121565199424,3cyclictest131rcu_preempt20:00:010
1215651994235,4cyclictest16950irq/224-800000019:40:010
1215651994219,17cyclictest275-21dbus-daemon19:50:010
1215661994110,7cyclictest131rcu_preempt19:50:012
121565499416,5cyclictest0-21swapper/100:05:141
1215654994129,9cyclictest23-21ksoftirqd/123:18:151
1215654994129,9cyclictest23-21ksoftirqd/123:18:141
1215654994127,8cyclictest310-21in:imklog22:41:301
121565499412,3cyclictest131rcu_preempt23:37:361
121565499412,3cyclictest131rcu_preempt20:25:031
1215654994121,5cyclictest23-21ksoftirqd/123:00:131
1215654994119,6cyclictest23-21ksoftirqd/123:23:141
1215654994117,5cyclictest23-21ksoftirqd/122:17:221
1215651994129,9cyclictest1219492-21sshd20:24:150
121565199411,11cyclictest1218464-21munin-run20:05:000
1215661994034,4cyclictest1219728-21kworker/2:220:52:122
121565499406,6cyclictest131rcu_preempt20:12:091
121565499404,11cyclictest131rcu_preempt00:15:391
121565499403,7cyclictest23-21ksoftirqd/120:40:021
1215654994029,8cyclictest1221565-21sshd21:04:141
1215654994021,4cyclictest23-21ksoftirqd/122:46:041
1215654994018,8cyclictest23-21ksoftirqd/122:57:111
121565499401,3cyclictest131rcu_preempt21:24:041
1215651994027,9cyclictest12-21ksoftirqd/019:30:010
1215651994026,11cyclictest1216361-21sshd19:21:140
1215651994025,11cyclictest3467602-21systemd-journal19:16:590
1215654993924,11cyclictest3467602-21systemd-journal19:15:011
1215654993922,11cyclictest275-21dbus-daemon23:30:011
1215654993920,15cyclictest23-21ksoftirqd/120:35:001
1215654993917,5cyclictest23-21ksoftirqd/119:50:151
1215654993917,4cyclictest23-21ksoftirqd/119:50:021
1215654993917,4cyclictest23-21ksoftirqd/119:36:001
1215651993927,9cyclictest1-21systemd19:35:000
121565199391,8cyclictest1218713-21latency_hist20:10:010
1215654993827,7cyclictest1221284-21sshd20:59:161
1215654993817,5cyclictest23-21ksoftirqd/120:47:361
1215654993817,5cyclictest23-21ksoftirqd/120:47:351
1215654993817,5cyclictest23-21ksoftirqd/119:27:061
1215651993825,10cyclictest1217907-21sshd19:52:150
1215651993824,8cyclictest1215656-21latency_hist19:10:020
1215654993719,5cyclictest23-21ksoftirqd/120:10:011
1215654993717,4cyclictest23-21ksoftirqd/120:01:041
1215654993715,6cyclictest131rcu_preempt20:44:591
1215651993724,10cyclictest275-21dbus-daemon20:15:010
121565199371,18cyclictest0-21swapper/020:55:020
1215654993624,9cyclictest3467602-21systemd-journal19:18:161
1215651993629,4cyclictest0-21swapper/020:37:580
1215651993626,8cyclictest12-21ksoftirqd/020:50:020
121565199361,31cyclictest0-21swapper/019:15:010
121565499354,4cyclictest131rcu_preempt20:55:001
121565499341,5cyclictest131rcu_preempt19:35:011
121565199341,3cyclictest131rcu_preempt20:50:020
1215654993216,13cyclictest23-21ksoftirqd/119:10:011
1215661993121,7cyclictest271irq_work/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional