You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-16 - 16:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sun Aug 16, 2026 12:45:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1383075998925,20cyclictest131rcu_preempt11:40:021
1383075998925,20cyclictest131rcu_preempt11:40:011
1383084997632,35cyclictest37-21ksoftirqd/310:05:023
1383079997246,9cyclictest30-21ksoftirqd/212:16:302
138307999705,10cyclictest30-21ksoftirqd/211:00:012
1383079996733,6cyclictest30-21ksoftirqd/209:30:002
1383075996733,10cyclictest23-21ksoftirqd/110:05:001
1383079996440,9cyclictest30-21ksoftirqd/212:38:342
1383075996417,12cyclictest23-21ksoftirqd/108:00:001
138307599641,14cyclictest23-21ksoftirqd/110:00:011
138307999637,10cyclictest30-21ksoftirqd/211:11:252
138307999634,11cyclictest30-21ksoftirqd/209:50:012
1383079996339,14cyclictest30-21ksoftirqd/211:23:332
1383075996328,7cyclictest23-21ksoftirqd/111:54:201
1383070996313,25cyclictest12-21ksoftirqd/010:52:410
138307599621,7cyclictest1405642-21modprobe09:45:011
138307999618,8cyclictest30-21ksoftirqd/211:45:002
138307999615,11cyclictest30-21ksoftirqd/210:10:002
1383075996119,11cyclictest23-21ksoftirqd/112:20:011
1383075996113,6cyclictest131rcu_preempt10:20:001
1383075996113,6cyclictest131rcu_preempt10:20:001
1383075996111,6cyclictest131rcu_preempt12:15:011
1383075996111,6cyclictest131rcu_preempt12:15:011
138307099614,50cyclictest1431389-21modprobe10:40:000
138307099613,37cyclictest131rcu_preempt09:48:190
138307999607,8cyclictest30-21ksoftirqd/211:30:032
138307599603,10cyclictest23-21ksoftirqd/112:40:001
1383070996027,16cyclictest1460360-21ssh11:41:560
138307999597,8cyclictest30-21ksoftirqd/212:05:002
138307599597,7cyclictest23-21ksoftirqd/109:20:011
1383070995927,15cyclictest1438934-21ssh10:55:500
1383070995919,35cyclictest12-21ksoftirqd/010:20:350
138307599586,8cyclictest23-21ksoftirqd/110:40:001
1383075995817,6cyclictest131rcu_preempt11:00:011
1383075995814,6cyclictest131rcu_preempt12:25:011
138307099585,3cyclictest0-21swapper/010:00:010
1383070995837,17cyclictest16850irq/134-800000012:05:010
1383070995830,12cyclictest1466960-21ssh11:55:560
1383070995829,13cyclictest1462841-21ssh11:47:230
1383070995829,13cyclictest1462841-21ssh11:47:220
1383070995826,15cyclictest1455208-21ssh11:30:530
138307099581,41cyclictest1434134-21ssh10:45:200
1383079995749,6cyclictest30-21ksoftirqd/212:15:022
1383079995749,6cyclictest30-21ksoftirqd/212:15:022
1383079995727,20cyclictest30-21ksoftirqd/208:10:012
1383079995713,6cyclictest131rcu_preempt11:20:002
1383079995710,7cyclictest131rcu_preempt10:40:012
138307599579,11cyclictest23-21ksoftirqd/109:50:261
138307599574,10cyclictest23-21ksoftirqd/111:45:001
138307599573,10cyclictest23-21ksoftirqd/107:25:011
138307599572,50cyclictest1-21systemd08:30:011
1383075995715,6cyclictest131rcu_preempt11:35:021
1383075995711,6cyclictest131rcu_preempt12:05:011
138307999567,8cyclictest30-21ksoftirqd/212:30:002
138307999566,5cyclictest131rcu_preempt08:50:022
138307599569,6cyclictest131rcu_preempt10:55:001
138307599567,6cyclictest131rcu_preempt10:30:021
138307599565,10cyclictest23-21ksoftirqd/112:30:011
138307599564,11cyclictest23-21ksoftirqd/111:10:011
138307599564,11cyclictest23-21ksoftirqd/111:10:011
1383075995612,6cyclictest131rcu_preempt08:25:011
1383075995611,5cyclictest131rcu_preempt11:27:381
1383075995611,5cyclictest131rcu_preempt11:27:381
1383075995611,21cyclictest131rcu_preempt10:46:241
1383070995628,12cyclictest1425807-21ssh10:27:520
138307099562,47cyclictest309-21in:imuxsock09:40:000
1383070995620,16cyclictest1475595-21ssh12:15:010
1383070995620,16cyclictest1475595-21ssh12:15:010
138307999557,5cyclictest131rcu_preempt11:50:022
138307999557,5cyclictest131rcu_preempt11:50:022
138307999556,8cyclictest30-21ksoftirqd/212:25:022
138307999553,11cyclictest30-21ksoftirqd/209:55:012
138307599559,6cyclictest131rcu_preempt10:45:011
138307599558,5cyclictest131rcu_preempt10:35:021
138307599556,6cyclictest131rcu_preempt08:05:031
138307599556,6cyclictest131rcu_preempt08:05:021
1383075995513,6cyclictest131rcu_preempt10:06:451
1383075995511,5cyclictest131rcu_preempt11:49:421
1383075995511,5cyclictest131rcu_preempt11:49:411
1383070995531,20cyclictest16850irq/134-800000009:30:030
1383070995531,20cyclictest16850irq/134-800000009:30:030
1383070995525,14cyclictest1418559-21ssh10:12:320
138307999549,6cyclictest30-21ksoftirqd/209:45:042
138307999548,9cyclictest30-21ksoftirqd/212:31:542
138307999544,9cyclictest1391576-21ssh09:14:592
1383079995444,6cyclictest1388806-21kworker/2:111:08:372
1383079995444,6cyclictest1388806-21kworker/2:111:08:362
138307999544,13cyclictest30-21ksoftirqd/209:25:012
1383079995428,16cyclictest30-21ksoftirqd/211:59:032
138307999541,7cyclictest0-21swapper/207:35:002
1383079995411,6cyclictest131rcu_preempt07:45:022
138307599549,6cyclictest131rcu_preempt09:36:161
138307599549,5cyclictest131rcu_preempt09:30:251
138307599549,5cyclictest131rcu_preempt09:30:241
138307599548,6cyclictest131rcu_preempt09:48:301
138307599547,8cyclictest23-21ksoftirqd/110:15:011
138307599545,5cyclictest131rcu_preempt11:15:011
138307599541,7cyclictest0-21swapper/108:20:021
1383075995413,6cyclictest131rcu_preempt12:30:171
138307099548,4cyclictest131rcu_preempt11:55:000
138307099547,7cyclictest131rcu_preempt08:50:010
1383070995447,5cyclictest19350irq/34-mmc108:25:020
1383070995434,15cyclictest16850irq/134-800000011:18:250
1383070995428,10cyclictest1392554-21ssh09:16:480
1383070995423,4cyclictest14545041ssh11:29:520
1383070995423,4cyclictest14545041ssh11:29:520
138307099542,33cyclictest131rcu_preempt12:31:030
138307099541,37cyclictest1412985-21ssh10:00:170
138307999537,8cyclictest30-21ksoftirqd/211:05:012
138307999535,10cyclictest30-21ksoftirqd/210:27:312
1383079995343,6cyclictest30-21ksoftirqd/211:39:572
1383079995343,6cyclictest30-21ksoftirqd/211:39:572
138307999533,14cyclictest30-21ksoftirqd/207:19:592
1383079995312,7cyclictest131rcu_preempt10:22:262
1383079995310,5cyclictest131rcu_preempt10:20:022
1383079995310,5cyclictest131rcu_preempt10:20:022
138307599536,5cyclictest131rcu_preempt12:07:121
138307599534,9cyclictest23-21ksoftirqd/109:30:021
1383075995318,8cyclictest23-21ksoftirqd/111:22:091
1383075995316,8cyclictest23-21ksoftirqd/107:55:011
1383075995315,5cyclictest131rcu_preempt10:21:561
1383075995314,10cyclictest23-21ksoftirqd/111:02:181
1383070995331,11cyclictest12-21ksoftirqd/012:25:020
138307099532,7cyclictest275-21dbus-daemon08:30:010
1383070995327,11cyclictest1397221-21ssh09:26:470
1383070995326,12cyclictest1416248-21ssh10:07:370
1383070995325,12cyclictest1486694-21ssh12:38:410
1383070995324,13cyclictest1394819-21ssh09:21:370
1383070995323,14cyclictest1403939-21ssh09:41:040
1383070995319,30cyclictest12-21ksoftirqd/011:05:140
1383070995319,30cyclictest12-21ksoftirqd/011:05:140
138307999529,5cyclictest131rcu_preempt10:01:342
138307999528,8cyclictest30-21ksoftirqd/207:50:012
138307999528,8cyclictest30-21ksoftirqd/207:50:012
138307999527,8cyclictest30-21ksoftirqd/210:35:022
138307999526,9cyclictest30-21ksoftirqd/209:10:012
138307999526,4cyclictest131rcu_preempt10:45:252
138307999525,5cyclictest131rcu_preempt10:00:002
138307999524,6cyclictest30-21ksoftirqd/211:54:592
1383079995244,5cyclictest1386141-21kworker/2:208:45:012
1383079995215,5cyclictest131rcu_preempt08:40:012
1383079995215,5cyclictest131rcu_preempt08:40:002
138307599529,7cyclictest131rcu_preempt09:00:011
138307599529,6cyclictest131rcu_preempt11:15:111
138307599529,5cyclictest131rcu_preempt08:10:021
138307599526,7cyclictest23-21ksoftirqd/108:45:001
138307599526,12cyclictest23-21ksoftirqd/107:20:011
138307599523,9cyclictest23-21ksoftirqd/109:23:591
1383075995212,5cyclictest131rcu_preempt07:15:001
138307099527,32cyclictest1383073-21latency_hist07:10:010
138307099523,42cyclictest1389313-21modprobe09:10:010
1383070995227,10cyclictest1452184-21ssh11:24:410
1383084995116,16cyclictest1376432-21kworker/u8:007:15:013
138307999517,10cyclictest30-21ksoftirqd/212:09:292
138307999514,9cyclictest30-21ksoftirqd/210:53:252
1383079995143,5cyclictest1388806-21kworker/2:110:40:262
1383079995142,6cyclictest1386141-21kworker/2:208:26:252
1383079995114,10cyclictest30-21ksoftirqd/210:15:022
138307599518,9cyclictest23-21ksoftirqd/111:56:431
138307599516,10cyclictest23-21ksoftirqd/107:40:021
1383075995110,5cyclictest131rcu_preempt09:12:031
1383070995126,12cyclictest1433338-21ssh10:44:060
138307099512,43cyclictest1422071-21systemd-run10:20:020
138307099512,43cyclictest1422071-21systemd-run10:20:020
138307999504,8cyclictest30-21ksoftirqd/209:18:002
138307999504,7cyclictest30-21ksoftirqd/207:25:012
1383079995041,6cyclictest1388806-21kworker/2:109:38:202
1383079995027,8cyclictest30-21ksoftirqd/209:32:572
1383079995027,8cyclictest30-21ksoftirqd/209:32:572
1383079995012,5cyclictest131rcu_preempt07:30:002
138307599509,5cyclictest131rcu_preempt08:40:011
138307599509,5cyclictest131rcu_preempt08:40:001
138307599509,5cyclictest131rcu_preempt07:45:021
138307599509,11cyclictest23-21ksoftirqd/107:10:031
138307599508,9cyclictest23-21ksoftirqd/107:35:031
1383070995021,13cyclictest1477716-21ssh12:19:350
138307099501,7cyclictest1447647-21modprobe11:15:000
138308499492,3cyclictest131rcu_preempt08:20:023
138307999494,9cyclictest30-21ksoftirqd/208:00:002
1383079994941,5cyclictest1388806-21kworker/2:111:26:102
1383079994941,5cyclictest1388806-21kworker/2:111:26:102
1383079994941,5cyclictest1373792-21kworker/2:008:55:492
1383070994934,3cyclictest0-21swapper/012:30:020
1383070994922,17cyclictest12-21ksoftirqd/007:35:000
1383070994920,14cyclictest1409726-21ssh09:53:390
1383084994834,10cyclictest37-21ksoftirqd/309:15:013
1383079994841,4cyclictest1376166-21kworker/2:207:14:032
1383079994840,5cyclictest1386141-21kworker/2:208:24:192
1383079994840,5cyclictest1373792-21kworker/2:009:00:462
1383079994840,5cyclictest1373792-21kworker/2:008:53:202
1383079994840,5cyclictest1373792-21kworker/2:008:53:192
1383079994840,5cyclictest1373792-21kworker/2:008:17:302
1383079994839,6cyclictest1373792-21kworker/2:008:10:342
138307599488,10cyclictest23-21ksoftirqd/108:35:031
138307599481,5cyclictest131rcu_preempt07:30:011
1383070994835,8cyclictest0-21swapper/007:40:010
138307099481,16cyclictest1459287-21munin-plugin-st11:40:000
138307099481,16cyclictest1459287-21munin-plugin-st11:39:590
1383084994733,10cyclictest275-21dbus-daemon12:15:013
1383084994733,10cyclictest275-21dbus-daemon12:15:013
138307999477,6cyclictest131rcu_preempt07:10:032
1383079994739,5cyclictest1384526-21kworker/2:108:00:312
1383079994739,5cyclictest1384526-21kworker/2:108:00:302
1383079994739,5cyclictest1384526-21kworker/2:107:50:362
1383075994711,6cyclictest131rcu_preempt09:10:021
138307099476,21cyclictest19350irq/34-mmc108:30:020
1383070994732,10cyclictest1-21systemd08:45:000
1383070994718,25cyclictest1388957-21date09:00:020
1383084994629,12cyclictest275-21dbus-daemon10:20:013
1383084994629,12cyclictest275-21dbus-daemon10:20:013
1383084994623,17cyclictest275-21dbus-daemon08:55:013
1383084994623,17cyclictest275-21dbus-daemon08:55:003
1383079994638,5cyclictest1386141-21kworker/2:208:30:092
1383079994638,5cyclictest1373792-21kworker/2:007:35:052
138307599467,8cyclictest23-21ksoftirqd/107:50:011
138307599467,8cyclictest23-21ksoftirqd/107:50:001
138307599466,6cyclictest131rcu_preempt09:05:011
138307599465,7cyclictest23-21ksoftirqd/108:50:011
1383075994633,10cyclictest1386346-21sshd08:10:261
138307099462,39cyclictest1389299-21diskmemload11:00:040
1383084994526,13cyclictest309-21in:imuxsock08:30:013
1383070994538,4cyclictest16850irq/134-800000010:35:010
1383070994529,13cyclictest1390187-21ssh09:11:420
1383070994527,15cyclictest1471412-21rm12:05:350
138308499444,5cyclictest131rcu_preempt11:55:383
1383084994431,8cyclictest3467602-21systemd-journal07:55:023
1383084994430,10cyclictest1-21systemd09:35:013
1383084994430,10cyclictest1-21systemd09:35:003
1383084994428,13cyclictest37-21ksoftirqd/312:05:173
1383084994428,12cyclictest275-21dbus-daemon09:55:013
1383084994411,4cyclictest131rcu_preempt09:10:013
138307099442,5cyclictest1-21systemd08:00:000
1383070994410,12cyclictest275-21dbus-daemon08:05:010
1383070994410,12cyclictest275-21dbus-daemon08:05:010
138308499434,4cyclictest131rcu_preempt08:50:003
1383084994332,7cyclictest309-21in:imuxsock11:55:013
1383084994328,12cyclictest1407895-21cron09:50:003
1383084994328,11cyclictest37-21ksoftirqd/310:40:003
1383084994326,11cyclictest275-21dbus-daemon12:20:013
138307599439,6cyclictest131rcu_preempt08:55:021
138307599439,6cyclictest131rcu_preempt08:55:021
138307099436,7cyclictest12-21ksoftirqd/008:15:020
1383070994327,11cyclictest1-21systemd07:15:010
138307099432,37cyclictest1385052-21latency_hist07:45:020
138308499429,6cyclictest131rcu_preempt09:42:243
1383084994230,10cyclictest37-21ksoftirqd/309:21:553
1383084994229,10cyclictest37-21ksoftirqd/311:43:243
1383084994228,9cyclictest37-21ksoftirqd/307:40:003
1383084994228,10cyclictest37-21ksoftirqd/312:40:003
1383084994228,10cyclictest3467602-21systemd-journal10:09:263
1383084994227,12cyclictest37-21ksoftirqd/311:48:293
1383084994227,12cyclictest37-21ksoftirqd/311:48:293
138308499421,3cyclictest131rcu_preempt12:34:283
1383070994228,9cyclictest1-21systemd07:20:000
1383084994127,11cyclictest37-21ksoftirqd/308:05:013
1383084994127,11cyclictest37-21ksoftirqd/308:05:013
138308499412,21cyclictest0-21swapper/311:08:023
138308499412,21cyclictest0-21swapper/311:08:023
1383084994122,15cyclictest275-21dbus-daemon11:00:013
138308499411,6cyclictest0-21swapper/311:02:143
1383084994116,17cyclictest37-21ksoftirqd/307:35:003
138308499411,3cyclictest131rcu_preempt09:59:593
138308499407,8cyclictest37-21ksoftirqd/311:25:503
138308499407,8cyclictest37-21ksoftirqd/311:25:503
138308499404,3cyclictest131rcu_preempt11:22:103
1383084994029,8cyclictest37-21ksoftirqd/312:04:423
1383084994028,10cyclictest37-21ksoftirqd/310:48:263
1383084994024,12cyclictest275-21dbus-daemon10:35:013
138308499402,3cyclictest131rcu_preempt11:33:353
1383084994023,11cyclictest3467602-21systemd-journal10:30:003
138308499401,5cyclictest131rcu_preempt09:20:023
1383070994027,8cyclictest12-21ksoftirqd/007:50:040
1383070994027,8cyclictest12-21ksoftirqd/007:50:030
1383084993929,7cyclictest1386383-21/usr/sbin/munin08:15:023
1383084993929,7cyclictest1386109-21sshd08:08:353
1383084993927,9cyclictest37-21ksoftirqd/309:28:433
1383084993925,10cyclictest3467602-21systemd-journal07:25:003
1383084993924,11cyclictest1-21systemd07:50:013
1383084993924,11cyclictest1-21systemd07:50:003
138308499391,3cyclictest131rcu_preempt09:05:023
1383070993933,3cyclictest16950irq/224-800000007:54:270
1383070993924,10cyclictest309-21in:imuxsock07:25:030
1383070993924,10cyclictest12-21ksoftirqd/008:39:320
1383070993924,10cyclictest12-21ksoftirqd/008:39:320
138308499385,6cyclictest131rcu_preempt11:20:023
1383084993827,9cyclictest37-21ksoftirqd/311:38:243
1383084993827,9cyclictest37-21ksoftirqd/311:38:243
1383084993825,9cyclictest0-21swapper/308:25:013
138308499382,4cyclictest131rcu_preempt10:54:493
1383084993822,13cyclictest0-21swapper/309:40:023
138308499381,3cyclictest131rcu_preempt12:27:493
1383084993811,6cyclictest131rcu_preempt10:11:223
138307099381,7cyclictest151rcuc/008:20:000
138308499374,8cyclictest37-21ksoftirqd/310:45:003
1383084993728,7cyclictest37-21ksoftirqd/307:20:023
1383084993728,6cyclictest37-21ksoftirqd/309:00:023
1383084993726,9cyclictest37-21ksoftirqd/312:21:133
1383084993726,9cyclictest37-21ksoftirqd/308:45:003
138308499372,3cyclictest131rcu_preempt10:25:013
1383070993722,10cyclictest309-21in:imuxsock07:20:020
138308499362,5cyclictest131rcu_preempt07:30:013
138308499362,3cyclictest131rcu_preempt11:14:103
1383084993621,11cyclictest1387509-21latency_hist08:35:023
138307099362,5cyclictest1388495-21sshd08:54:270
138307099362,5cyclictest1388495-21sshd08:54:260
1383070993622,10cyclictest1389026-21sshd09:03:480
1383070993622,10cyclictest12-21ksoftirqd/008:06:000
138308499351,5cyclictest131rcu_preempt07:45:003
1383084993424,7cyclictest1387694-21sshd08:36:263
1383084993424,7cyclictest1387694-21sshd08:36:253
138308499341,3cyclictest131rcu_preempt07:10:023
138308499313,3cyclictest131rcu_preempt07:55:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional