You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-12 - 14:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Fri Jun 12, 2026 00:45:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1330104998424,56cyclictest1355236-21ssh21:51:460
1330104998328,51cyclictest1418152-21ssh00:11:340
1330104998115,13cyclictest1349548-21ssh21:39:290
1330104998115,13cyclictest1349548-21ssh21:39:290
133010499801,13cyclictest1420063-21ssh00:15:460
1330104997829,43cyclictest1335788-21run-parts21:00:000
1330104997636,21cyclictest16850irq/134-800000023:26:250
1330104997623,49cyclictest1427912-21ssh00:33:120
1330104997617,12cyclictest1368405-21ssh22:20:490
1330104997424,45cyclictest1383583-21ssh22:54:550
133010499742,15cyclictest1379064-21ssh22:44:500
1330104997316,38cyclictest364-21snmpd21:50:020
1330104997227,41cyclictest1391296-21sh23:11:490
1330104997215,14cyclictest1407179-21ssh23:47:160
1330104997123,30cyclictest1387093-21ssh23:02:320
133010499711,64cyclictest1341567-21sshd21:21:200
1330104997028,24cyclictest1405644-21ssh23:44:010
1330104997021,32cyclictest1429680-21ssh00:37:090
1330104996927,26cyclictest1389364-21ssh23:07:350
1330104996823,40cyclictest1363775-21ssh22:10:400
1330104996816,47cyclictest1384498-21ssh22:56:390
1330104996816,47cyclictest1384498-21ssh22:56:390
133010499672,7cyclictest1412817-21systemctl00:00:000
1330104996724,27cyclictest1360986-21ssh22:04:510
133010499672,15cyclictest1334003-21munin-run20:25:010
133010499672,13cyclictest1351705-21ssh21:44:100
1330104996720,43cyclictest364-21snmpd20:19:490
133010499669,13cyclictest1344473-21ssh21:27:590
1330104996640,21cyclictest1376926-21systemd-run22:40:010
133010499661,15cyclictest48-21kcompactd000:00:040
1330104996442,5cyclictest16850irq/134-800000000:10:000
1330104996428,32cyclictest1370861-21ssh22:26:160
133010499641,17cyclictest1366374-21ssh22:16:260
133010499641,17cyclictest1366374-21ssh22:16:250
1330111996343,14cyclictest275-21dbus-daemon23:59:592
1330104996334,24cyclictest16850irq/134-800000023:21:260
133010499632,14cyclictest1331893-21sshd19:44:200
1330104996320,27cyclictest1358574-21ssh21:59:310
1330104996318,41cyclictest150-21pr/ttyS200:20:470
133010499631,58cyclictest0-21swapper/021:12:280
1330104996311,11cyclictest1372925-21ssh22:30:490
1330109996229,8cyclictest23-21ksoftirqd/123:27:351
133010499621,56cyclictest0-21swapper/022:08:500
133010499621,56cyclictest0-21swapper/022:08:500
1330109996116,10cyclictest23-21ksoftirqd/123:30:111
1330109996115,5cyclictest131rcu_preempt21:39:341
1330109996115,5cyclictest131rcu_preempt21:39:341
133010499612,54cyclictest1-21systemd21:35:010
1330104996119,38cyclictest0-21swapper/020:26:180
133010499611,4cyclictest131rcu_preempt23:17:490
133010499611,15cyclictest1338918-21ssh21:15:170
1330109996017,6cyclictest131rcu_preempt22:46:201
1330109996017,6cyclictest131rcu_preempt22:46:201
133010999601,12cyclictest23-21ksoftirqd/122:40:011
1330104996037,20cyclictest16950irq/224-800000023:35:020
133010499602,3cyclictest0-21swapper/000:30:000
133010499602,3cyclictest0-21swapper/000:30:000
1330104996020,36cyclictest0-21swapper/022:47:360
1330104996020,36cyclictest0-21swapper/022:47:360
133010499601,15cyclictest1331127-21munin-run19:30:000
133010499601,13cyclictest1331343-21sshd19:30:200
1330111995921,11cyclictest30-21ksoftirqd/200:05:012
133010999597,5cyclictest131rcu_preempt21:00:011
133010999595,9cyclictest23-21ksoftirqd/119:10:011
1330109995917,10cyclictest23-21ksoftirqd/122:32:171
133010499598,46cyclictest0-21swapper/021:03:330
133010499591,3cyclictest0-21swapper/020:09:200
1330104995913,13cyclictest1408599-21ssh23:50:170
1330111995841,8cyclictest1335833-21kworker/2:223:19:252
1330109995816,19cyclictest23-21ksoftirqd/121:15:021
133010999581,12cyclictest1403794-21modprobe23:40:001
1330104995831,22cyclictest16850irq/134-800000021:10:000
133010499582,14cyclictest364-21snmpd20:30:420
133010499581,13cyclictest364-21snmpd20:48:090
133010999574,9cyclictest23-21ksoftirqd/100:25:011
1330114995610,38cyclictest361rcuc/300:00:033
1330111995646,6cyclictest1317218-21kworker/2:020:15:042
133011199562,5cyclictest131rcu_preempt22:40:012
1330109995612,10cyclictest23-21ksoftirqd/121:25:021
133010499564,8cyclictest1331682-21wc19:40:010
133010499562,10cyclictest1-21systemd19:47:020
1330111995547,5cyclictest1374669-21kworker/2:300:25:342
1330111995547,5cyclictest1374669-21kworker/2:300:25:332
1330111995545,6cyclictest1317218-21kworker/2:021:15:282
133010999554,18cyclictest23-21ksoftirqd/122:54:291
133010499553,14cyclictest1332673-21munin-run20:00:000
1330104995523,28cyclictest12-21ksoftirqd/020:50:030
1330111995445,5cyclictest1358850-21kworker/2:122:14:592
1330111995441,6cyclictest1374669-21kworker/2:323:54:522
1330111995439,7cyclictest1374669-21kworker/2:300:39:312
1330109995446,6cyclictest23-21ksoftirqd/121:49:071
1330109995416,6cyclictest131rcu_preempt23:12:431
1330109995416,5cyclictest131rcu_preempt00:38:271
1330109995414,10cyclictest23-21ksoftirqd/123:01:261
1330109995413,5cyclictest131rcu_preempt22:59:341
1330109995413,5cyclictest131rcu_preempt22:59:341
1330109995412,5cyclictest131rcu_preempt23:25:021
133010499542,16cyclictest1335085-21cat20:45:020
133010499542,13cyclictest1330541-21cat19:15:020
133010499541,49cyclictest0-21swapper/023:32:150
1330111995345,5cyclictest1317218-21kworker/2:021:36:372
1330111995345,5cyclictest1317218-21kworker/2:021:36:362
1330111995345,5cyclictest1317218-21kworker/2:021:05:052
1330111995345,5cyclictest1317218-21kworker/2:020:52:152
1330111995343,7cyclictest1374669-21kworker/2:300:05:232
1330111995343,6cyclictest1317218-21kworker/2:020:50:012
1330111995339,6cyclictest1335833-21kworker/2:222:54:542
1330111995311,9cyclictest1317218-21kworker/2:019:40:012
133010999531,8cyclictest23-21ksoftirqd/100:08:531
1330109995317,5cyclictest131rcu_preempt23:44:591
1330109995315,10cyclictest23-21ksoftirqd/121:53:281
1330111995244,5cyclictest1412878-21kworker/2:100:19:002
1330111995244,5cyclictest1358850-21kworker/2:122:15:142
1330111995244,5cyclictest1358850-21kworker/2:122:15:142
1330111995243,6cyclictest1317218-21kworker/2:021:48:062
1330111995243,6cyclictest1317218-21kworker/2:021:28:142
133010999529,6cyclictest131rcu_preempt23:54:451
133010999528,6cyclictest131rcu_preempt21:32:261
133010999528,12cyclictest23-21ksoftirqd/122:40:221
1330109995234,14cyclictest3467602-21systemd-journal00:17:001
1330109995215,6cyclictest131rcu_preempt22:00:001
1330109995214,6cyclictest131rcu_preempt22:21:041
1330109995214,6cyclictest131rcu_preempt22:05:001
1330109995212,10cyclictest23-21ksoftirqd/100:00:001
133010999521,20cyclictest0-21swapper/122:07:291
133010999521,20cyclictest0-21swapper/122:07:291
1330109995210,7cyclictest131rcu_preempt23:19:321
133010499528,6cyclictest0-21swapper/019:25:000
1330111995143,5cyclictest1335833-21kworker/2:223:32:072
1330111995143,5cyclictest1335833-21kworker/2:222:32:392
1330111995142,6cyclictest1335833-21kworker/2:222:05:492
1330111995142,6cyclictest1335833-21kworker/2:222:05:492
1330111995142,6cyclictest1317218-21kworker/2:021:12:202
1330111995142,5cyclictest1335833-21kworker/2:222:44:042
1330111995138,6cyclictest1335833-21kworker/2:222:48:142
1330111995138,6cyclictest1335833-21kworker/2:222:48:142
133010999516,8cyclictest23-21ksoftirqd/120:45:001
1330109995115,7cyclictest23-21ksoftirqd/119:50:021
1330109995113,5cyclictest131rcu_preempt21:23:331
1330109995112,5cyclictest131rcu_preempt22:25:201
1330109995112,5cyclictest131rcu_preempt20:10:021
1330114995037,9cyclictest1423249-21sshd00:22:573
1330111995042,5cyclictest1374669-21kworker/2:323:42:512
1330111995042,5cyclictest1374669-21kworker/2:323:36:152
1330111995042,5cyclictest1374669-21kworker/2:323:02:292
1330111995042,5cyclictest1374669-21kworker/2:300:14:292
1330111995042,5cyclictest1335833-21kworker/2:223:07:102
1330111995042,5cyclictest1335833-21kworker/2:222:02:062
1330111995042,5cyclictest1335833-21kworker/2:221:56:142
1330111995042,5cyclictest1317218-21kworker/2:021:21:202
1330111995041,6cyclictest1374669-21kworker/2:323:46:142
1330111995041,6cyclictest1374669-21kworker/2:323:11:252
1330111995041,6cyclictest1317218-21kworker/2:021:44:262
1330111995041,6cyclictest1317218-21kworker/2:021:34:272
1330111995041,5cyclictest1374669-21kworker/2:300:33:412
1330111995041,5cyclictest1335833-21kworker/2:223:29:042
1330111995041,5cyclictest1335833-21kworker/2:222:26:462
133010999507,5cyclictest131rcu_preempt22:12:521
1330109995014,8cyclictest23-21ksoftirqd/123:50:011
1330109995013,10cyclictest23-21ksoftirqd/100:26:251
1330109995013,10cyclictest23-21ksoftirqd/100:26:241
1330109995012,6cyclictest131rcu_preempt21:44:051
1330109995010,5cyclictest131rcu_preempt23:07:401
1330109995010,5cyclictest131rcu_preempt00:10:381
133010499502,13cyclictest1330279-21latency_hist19:10:020
1330111994942,4cyclictest1412878-21kworker/2:100:21:182
1330111994941,5cyclictest1335833-21kworker/2:223:23:052
1330111994941,5cyclictest1329699-21kworker/2:119:48:202
1330111994941,5cyclictest1317218-21kworker/2:021:51:112
1330111994941,5cyclictest1317218-21kworker/2:020:35:062
1330111994940,5cyclictest1374669-21kworker/2:322:59:472
1330111994940,5cyclictest1374669-21kworker/2:322:59:472
1330111994940,5cyclictest1335833-21kworker/2:222:20:362
133010999496,11cyclictest23-21ksoftirqd/100:05:011
1330109994910,6cyclictest131rcu_preempt19:38:201
133010499493,6cyclictest1330587-21modprobe19:20:010
133010499492,11cyclictest1332543-21latency_hist19:55:020
1330114994814,5cyclictest346771850lldpd23:01:293
1330111994840,5cyclictest1317218-21kworker/2:020:25:112
1330111994840,5cyclictest1317218-21kworker/2:019:15:002
1330111994831,13cyclictest1331345-21sshd19:30:202
133010999483,10cyclictest23-21ksoftirqd/119:20:021
1330109994814,5cyclictest131rcu_preempt20:25:201
1330109994811,5cyclictest131rcu_preempt21:15:451
133010499484,9cyclictest1334754-21modprobe20:40:000
133010499481,13cyclictest1333036-21latency_hist20:05:020
1330114994735,6cyclictest1336317-21diskmemload21:10:003
1330111994739,5cyclictest1329699-21kworker/2:119:50:202
1330111994739,5cyclictest1329699-21kworker/2:119:41:212
1330111994739,5cyclictest1317218-21kworker/2:020:33:212
1330111994739,5cyclictest1317218-21kworker/2:019:15:472
1330111994729,14cyclictest3467602-21systemd-journal21:00:002
133010999474,17cyclictest23-21ksoftirqd/100:33:281
1330109994734,9cyclictest3467602-21systemd-journal20:54:201
133010499473,13cyclictest1333437-21cron20:15:000
133011499463,3cyclictest131rcu_preempt22:26:263
1330114994633,8cyclictest1403813-21sendmail-msp23:40:003
1330114994626,11cyclictest3467602-21systemd-journal21:50:003
1330111994638,5cyclictest1317218-21kworker/2:020:20:032
1330111994638,5cyclictest1317218-21kworker/2:020:06:212
1330111994638,5cyclictest1317218-21kworker/2:020:03:272
1330111994638,5cyclictest1317218-21kworker/2:019:57:442
1330111994637,5cyclictest1317218-21kworker/2:021:02:432
133010999463,9cyclictest23-21ksoftirqd/120:00:001
1330109994637,6cyclictest23-21ksoftirqd/122:17:371
1330109994637,6cyclictest23-21ksoftirqd/122:17:371
1330109994631,10cyclictest309-21in:imuxsock20:45:021
1330114994532,10cyclictest1393417-21sshd23:16:253
1330114994530,11cyclictest37-21ksoftirqd/300:01:003
1330111994538,5cyclictest1317218-21kworker/2:020:18:532
1330111994538,4cyclictest1317218-21kworker/2:019:28:152
1330111994537,6cyclictest1317218-21kworker/2:020:42:382
1330109994533,9cyclictest23-21ksoftirqd/119:25:011
1330109994512,5cyclictest131rcu_preempt19:10:581
1330114994429,11cyclictest275-21dbus-daemon21:35:023
1330111994436,5cyclictest1317218-21kworker/2:019:22:272
133010999448,5cyclictest131rcu_preempt20:15:011
1330109994434,7cyclictest23-21ksoftirqd/121:02:191
133010999441,8cyclictest0-21swapper/120:40:011
133011499433,3cyclictest131rcu_preempt23:47:563
133011499433,20cyclictest1374838-21ssh22:35:023
1330114994329,10cyclictest275-21dbus-daemon19:30:013
1330114994328,11cyclictest1334733-21cron20:40:003
1330114994325,13cyclictest1333722-21systemctl20:20:013
133011499432,3cyclictest131rcu_preempt22:58:563
133011499432,3cyclictest131rcu_preempt22:58:553
133011499432,3cyclictest131rcu_preempt21:52:423
133011499432,36cyclictest37-21ksoftirqd/321:10:103
1330114994323,15cyclictest275-21dbus-daemon00:35:003
133011499431,3cyclictest131rcu_preempt22:40:183
133010999434,5cyclictest23-21ksoftirqd/119:55:001
1330109994324,13cyclictest3467602-21systemd-journal19:30:001
133011499423,3cyclictest131rcu_preempt23:42:443
1330114994230,8cyclictest37-21ksoftirqd/321:38:283
1330114994230,8cyclictest37-21ksoftirqd/321:38:283
133011499422,9cyclictest1401525-21munin-run23:35:003
1330114994229,10cyclictest37-21ksoftirqd/322:15:383
1330114994229,10cyclictest37-21ksoftirqd/322:15:383
133011499422,3cyclictest131rcu_preempt00:35:183
1330109994212,6cyclictest131rcu_preempt21:10:001
133011499416,7cyclictest37-21ksoftirqd/323:14:533
133011499414,3cyclictest131rcu_preempt22:13:093
1330114994132,4cyclictest37-21ksoftirqd/322:05:013
1330114994128,10cyclictest37-21ksoftirqd/320:50:013
1330114994126,10cyclictest37-21ksoftirqd/319:25:003
133010999411,12cyclictest23-21ksoftirqd/119:45:001
133011499404,4cyclictest131rcu_preempt23:55:013
1330114994029,8cyclictest37-21ksoftirqd/320:15:003
1330114994028,9cyclictest1388475-21sshd23:05:253
1330114994027,10cyclictest1376940-21latency_hist22:40:013
133011499401,3cyclictest131rcu_preempt00:20:003
1330114994011,5cyclictest131rcu_preempt00:11:203
1330111994016,19cyclictest131rcu_preempt19:10:012
133010999406,10cyclictest23-21ksoftirqd/120:05:011
1330109994028,10cyclictest23-21ksoftirqd/120:20:021
1330114993930,5cyclictest1415287-21sh00:05:103
1330114993927,10cyclictest37-21ksoftirqd/322:05:493
1330114993927,10cyclictest37-21ksoftirqd/322:05:493
1330114993926,10cyclictest37-21ksoftirqd/322:47:583
1330114993926,10cyclictest37-21ksoftirqd/322:47:573
1330114993925,11cyclictest37-21ksoftirqd/300:25:453
1330114993925,11cyclictest37-21ksoftirqd/300:25:453
133011499392,3cyclictest131rcu_preempt19:45:023
1330114993918,16cyclictest310-21in:imklog20:00:003
133011499391,5cyclictest0-21swapper/321:41:393
133011499384,3cyclictest131rcu_preempt21:22:023
133011499383,3cyclictest131rcu_preempt22:54:523
1330114993827,8cyclictest37-21ksoftirqd/321:59:103
133011499382,3cyclictest131rcu_preempt23:25:523
1330114993810,5cyclictest131rcu_preempt23:21:283
1330114993728,7cyclictest37-21ksoftirqd/322:23:103
1330114993725,9cyclictest1-21systemd19:50:013
133011499371,6cyclictest0-21swapper/321:28:353
133011499371,3cyclictest131rcu_preempt21:18:133
133011499371,3cyclictest131rcu_preempt20:25:003
1330109993728,7cyclictest23-21ksoftirqd/120:23:201
1330114993626,8cyclictest37-21ksoftirqd/320:30:023
1330114993624,10cyclictest37-21ksoftirqd/319:20:013
133010999365,5cyclictest131rcu_preempt19:32:041
133011499353,6cyclictest131rcu_preempt20:05:023
1330114993526,7cyclictest37-21ksoftirqd/321:00:023
1330114993525,5cyclictest309-21in:imuxsock20:42:143
1330114993522,9cyclictest37-21ksoftirqd/319:40:013
1330114993518,13cyclictest1-21systemd19:35:013
1330109993518,12cyclictest275-21dbus-daemon20:35:011
133011499345,6cyclictest131rcu_preempt21:02:203
133011499341,4cyclictest346771550lldpd20:30:243
1330114993321,8cyclictest37-21ksoftirqd/319:10:023
133011499331,3cyclictest131rcu_preempt19:51:063
1330114993219,10cyclictest3467602-21systemd-journal20:09:593
1330114993122,5cyclictest1335628-21latency_hist20:55:013
1330114993118,10cyclictest1330319-21cron19:15:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional