You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-23 - 18:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Sun Aug 23, 2026 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
129044021900,2sleep3575-21kworker/3:2-events09:42:433
11791642660,2sleep3575-21kworker/3:2+events07:47:503
11791642660,2sleep3575-21kworker/3:2+events07:47:503
11608212650,1sleep3471rcuc/307:32:413
1137786996240,19cyclictest1154723-21latency_hist07:25:010
1137786995851,5cyclictest1179725-21latency_hist07:50:000
1137786995851,5cyclictest1179725-21latency_hist07:50:000
1137786995531,21cyclictest1159636-21latency_hist07:30:010
11349222540,9chrt13-21ksoftirqd/007:07:420
1137786995343,8cyclictest1467017-21latency_hist12:40:010
1137790995040,7cyclictest1323335-21latency_hist10:15:011
1137790994739,5cyclictest1288714-21latency_hist09:40:011
1137790994737,7cyclictest1363110-21latency_hist10:55:011
1137790994636,8cyclictest1179008-21ntpq07:47:461
1137790994636,8cyclictest1179008-21ntpq07:47:461
1137790994433,9cyclictest1367341-21ntpq10:57:491
1137786994419,10cyclictest275-21systemd-journal11:00:000
1137790994331,8cyclictest275-21systemd-journal12:32:391
1137786994334,6cyclictest1437309-21latency_hist12:10:010
1137786994317,23cyclictest0-21swapper/010:52:340
12547592420,3sleep01137778-21cyclictest09:07:370
1137790994233,7cyclictest1273122-21ntpq09:22:501
1137790994232,7cyclictest1407605-21latency_hist11:40:011
1137786994221,19cyclictest1299419-21fschecks_time09:52:410
1137795994122,17cyclictest275-21systemd-journal12:20:002
1137790994135,4cyclictest1308538-21kworker/u8:2+flush-8:010:42:531
1137786994021,17cyclictest275-21systemd-journal12:24:370
1137786994020,18cyclictest1264951-21sort09:17:420
1137786993931,6cyclictest1209795-21/usr/sbin/munin08:22:470
1137786993922,15cyclictest1229151-21apt-key08:40:000
1137786993921,16cyclictest0-21swapper/008:45:010
1137786993920,17cyclictest1349261-21sort10:42:410
1137786993919,2cyclictest0-21swapper/010:35:010
1137786993914,6cyclictest1223944-21unixbench_singl08:32:520
1137795993821,2cyclictest1234855-21aten2_r5power_v08:47:372
113779599381,8cyclictest11608252sed07:32:412
1137790993827,8cyclictest1343120-21latency_hist10:35:011
1137786993831,5cyclictest1189383-21grep07:57:540
1137786993821,16cyclictest1332995-21unixbench_singl10:22:550
1137786993821,15cyclictest1443283-21sort12:17:410
1137786993821,15cyclictest0-21swapper/007:17:340
1137786993819,2cyclictest0-21swapper/007:42:430
1137786993818,3cyclictest0-21swapper/007:32:450
113779599379,10cyclictest1209820-21apt-get08:22:362
1137795993722,13cyclictest1205083-21cpuspeed_turbos08:17:372
1137790993732,3cyclictest1100263-21kworker/u8:2+events_unbound08:27:391
1137786993720,2cyclictest1382624-21unixbench_multi11:12:540
1137786993718,2cyclictest0-21swapper/010:47:450
1137786993717,2cyclictest0-21swapper/009:15:000
1137786993716,3cyclictest0-21swapper/012:00:000
1137786993716,3cyclictest0-21swapper/012:00:000
1137786993714,6cyclictest1387826-21latency_hist11:20:010
1137786993711,22cyclictest1169511-21kworker/0:1+events07:52:360
1137795993629,5cyclictest1100263-21kworker/u8:2+flush-8:009:02:522
1137790993629,5cyclictest1466754-21grep12:37:551
1137786993627,7cyclictest1219235-21latency_hist08:30:010
1137786993622,12cyclictest850-21mta-sts-daemon12:31:300
1137786993615,4cyclictest1286743-21kthreadcore09:37:440
113779599352,13cyclictest1190062-21apt-get08:02:362
113778699353,9cyclictest0-21swapper/008:52:070
1137786993529,4cyclictest1397712-21unin-run11:30:000
113778699352,14cyclictest573-21runrttasks11:02:350
1137786993518,2cyclictest0-21swapper/009:02:440
1137790993427,5cyclictest1110255-21kworker/u8:3+flush-8:008:32:371
1137790993426,5cyclictest275-21systemd-journal07:10:331
1137786993417,3cyclictest0-21swapper/008:57:440
1137786993416,16cyclictest0-21swapper/010:37:440
1137786993415,4cyclictest1377939-21cron11:10:000
1137786993414,3cyclictest0-21swapper/009:32:390
113778699341,32cyclictest1204448-21kworker/0:2+events08:16:360
1137795993323,8cyclictest1348277-21/usr/sbin/munin10:40:012
1137790993327,4cyclictest1413023-21apt-get11:47:351
1137790993327,4cyclictest1259031-21kworker/u8:3+flush-8:010:50:011
1137790993327,4cyclictest1259031-21kworker/u8:3+flush-8:010:17:421
1137790993327,4cyclictest1100263-21kworker/u8:2+flush-8:009:27:391
1137790993326,5cyclictest275-21systemd-journal08:23:341
113778699332,9cyclictest0-21swapper/009:58:160
113778699332,9cyclictest0-21swapper/009:58:160
1137786993328,4cyclictest1259031-21kworker/u8:3+flush-8:010:02:550
1137786993324,6cyclictest1328251-21latency_hist10:20:010
11482492320,5sleep30-21swapper/307:17:443
1137790993227,3cyclictest1457116-21systemd-run12:30:011
1137790993226,4cyclictest1442261-21kworker/u8:1+events_unbound12:17:431
1137790993226,4cyclictest1279290-21apt-get09:32:341
1137790993226,4cyclictest1158930-21ntpq07:27:501
1137790993226,4cyclictest1110255-21kworker/u8:3+flush-8:008:47:391
1137790993223,7cyclictest1292923-21ntpq09:42:471
1137790993222,8cyclictest1307732-21ntpq09:57:481
1137790993222,8cyclictest1307732-21ntpq09:57:481
1137790993222,7cyclictest1382885-21latency_hist11:15:011
1137790993221,8cyclictest275-21systemd-journal10:25:331
113778699327,10cyclictest1401882-21grep11:32:470
1137786993210,2cyclictest0-21swapper/011:45:430
1137790993127,2cyclictest849-21mta-sts-daemon08:00:001
1137790993126,3cyclictest1348262-21awk10:40:001
1137786993114,4cyclictest0-21swapper/009:22:380
1137795993019,4cyclictest1343142-21gpgv10:35:002
1137790993026,3cyclictest1249162-21systemd-run09:00:001
1137790993026,2cyclictest1428088-21cpuspeed_turbos12:02:361
1137790993025,4cyclictest1100263-21kworker/u8:2+events_unbound08:37:541
1137790993024,4cyclictest1137635-21kworker/u8:1+flush-8:007:42:531
1137790993024,4cyclictest1100263-21kworker/u8:2+flush-8:008:02:401
1137790993023,6cyclictest1332939-21unixbench_multi10:22:551
1137790993022,6cyclictest1303269-21grep09:52:561
1137786993023,5cyclictest1110255-21kworker/u8:3+flush-8:007:10:100
1137795992925,2cyclictest1165134-21aten2_r5power_v07:37:382
1137790992924,3cyclictest1427410-21idleruntime-cro12:00:001
1137790992924,3cyclictest1427410-21idleruntime-cro12:00:001
1137790992922,5cyclictest1268221-21ntpq09:17:511
1137790992919,8cyclictest1421731-21ntpq11:52:491
1137786992921,6cyclictest1169561-21latency_hist07:40:000
1137786992914,3cyclictest0-21swapper/012:02:350
113778699291,26cyclictest1318438-21kworker/0:2+events11:52:500
113778699291,23cyclictest1268940-21kworker/0:0+events10:11:220
1137795992824,3cyclictest1274395-21aten2_r5power_p09:27:362
1137795992824,3cyclictest1184397-21unixbench_multi07:52:562
1137795992820,6cyclictest1229108-21apt-get08:40:002
1137790992824,3cyclictest1336768-21kworker/u8:1+flush-8:011:20:051
1137790992822,4cyclictest1154742-21systemd-run07:25:001
1137790992820,6cyclictest1204893-21grep08:17:361
1137790992820,5cyclictest275-21systemd-journal10:03:331
1137786992823,3cyclictest1291351-21kthreadcore09:42:440
1137786992816,9cyclictest275-21systemd-journal11:45:000
136000022716,6sleep30-21swapper/310:52:433
1137795992723,2cyclictest1230025-21cpuspeed_turbos08:42:372
1137790992722,3cyclictest1100263-21kworker/u8:2+events_unbound09:12:521
1137790992721,4cyclictest1283591-21kworker/u8:1+flush-8:009:47:531
1137790992719,6cyclictest1184659-21latency_hist07:55:001
1137790992715,9cyclictest275-21systemd-journal08:15:011
1137795992623,2cyclictest1279580-21fschecks_count09:32:382
1137795992622,2cyclictest1413337-21fschecks_time11:47:392
1137795992621,3cyclictest1263713-21unixbench_multi09:12:552
1137795992618,6cyclictest275-21systemd-journal11:03:332
1137790992620,5cyclictest1336768-21kworker/u8:1+flush-8:011:07:231
1137790992620,5cyclictest1137635-21kworker/u8:1+events_unbound08:40:011
1137790992620,3cyclictest0-21swapper/110:07:331
1137790992619,5cyclictest1397379-21kworker/u8:2+events_unbound11:42:371
1137790992619,5cyclictest1249587-21apt-get09:02:341
113778699268,17cyclictest1318438-21kworker/0:2+events10:27:540
1137786992620,4cyclictest1407386-21unixbench_singl11:37:540
1137795992522,2cyclictest1150329-21aten2_r5power_v07:22:372
1137790992521,2cyclictest1336768-21kworker/u8:1-events_unbound11:05:011
1137790992520,4cyclictest1100263-21kworker/u8:2+flush-8:007:37:561
1137790992520,3cyclictest1259031-21kworker/u8:3+flush-8:012:07:531
1137790992518,5cyclictest1353235-21kworker/u8:0+events_unbound12:12:361
1137790992516,7cyclictest1353235-21kworker/u8:0+flush-8:011:17:501
1137786992518,5cyclictest1278568-21grep09:27:540
113778699251,23cyclictest1169511-21kworker/0:1+events08:12:580
113779599240,9cyclictest1319216-21tune2fs10:12:392
1137790992420,3cyclictest1353235-21kworker/u8:0+flush-8:012:22:401
1137790992419,4cyclictest1137635-21kworker/u8:1+flush-8:008:52:521
1137790992415,7cyclictest1401932-21ntpq11:32:481
1137786992420,3cyclictest1453134-21sort12:27:410
1137786992420,2cyclictest0-21swapper/008:07:360
1137786992419,4cyclictest1442045-21unixbench_singl12:12:560
113779599237,4cyclictest0-21swapper/211:42:502
113779599237,3cyclictest12547602df09:07:372
1137795992320,2cyclictest1174295-21unixbench_multi07:42:542
1137795992319,3cyclictest1244898-21fschecks_count08:57:382
1137795992318,3cyclictest1363846-21fschecks_count10:57:402
1137795992312,9cyclictest275-21systemd-journal09:41:332
1137790992319,2cyclictest0-21swapper/109:07:451
1137790992319,2cyclictest0-21swapper/107:15:011
1137790992318,4cyclictest1137635-21kworker/u8:1+flush-8:008:07:041
113779599222,14cyclictest1308959-21apt-get10:02:362
1137795992218,2cyclictest1452630-21aten2_r5power_c12:27:362
1137795992218,2cyclictest1402376-21unixbench_multi11:32:542
1137795992217,4cyclictest1259031-21kworker/u8:3+events_unbound10:55:002
1137790992217,3cyclictest1137635-21kworker/u8:1+flush-8:007:32:541
1137786992218,3cyclictest0-21swapper/008:47:350
1137786992218,2cyclictest0-21swapper/009:50:000
1137786992217,3cyclictest0-21swapper/010:07:410
1137786992217,3cyclictest0-21swapper/008:05:010
11996992210,2chrt575-21kworker/3:2-events08:10:013
113779599212,13cyclictest1299072-21apt-get09:52:382
1137795992118,2cyclictest0-21swapper/211:37:412
1137795992117,2cyclictest1327984-21unixbench_multi10:17:542
1137795992117,2cyclictest1195212-21cpuspeed_turbos08:07:372
11330902214,11sleep10-21swapper/107:07:001
1137795992017,2cyclictest1373552-21aten2_r5power_p11:07:372
1137795992017,2cyclictest1240027-21fschecks_time08:52:392
1137795992016,3cyclictest1426571-21tail11:57:462
1137795992016,3cyclictest1426571-21tail11:57:462
1137795992014,4cyclictest1288486-21unixbench_singl09:37:542
1137790992016,2cyclictest0-21swapper/111:27:481
1137786992011,7cyclictest1388580-21grep11:22:410
11330662204,12sleep20-21swapper/207:06:412
113779599195,12cyclictest1352013-21tr10:42:452
113779599194,9cyclictest1421789-21ntpq11:52:512
113779599192,12cyclictest1388291-21apt-get11:22:382
113779599192,12cyclictest1333650-21apt-get10:27:362
113779599192,12cyclictest1219682-21apt-get08:32:332
113779599192,12cyclictest1185106-21apt-get07:57:362
1137795991915,3cyclictest1314205-21fschecks_count10:07:382
1137795991915,2cyclictest1142389-21grep07:12:532
1137795991915,2cyclictest0-21swapper/210:47:482
1137795991914,3cyclictest1432854-21aten2_r5power_c12:07:362
113779599191,13cyclictest1427860-21apt-get12:02:352
113779599191,13cyclictest1294134-21apt-get09:47:352
11326982193,7sleep30-21swapper/307:05:013
113779599184,12cyclictest1264638-21cpuspeed_turbos09:17:382
113779599182,12cyclictest1383336-21apt-get11:17:362
1137795991814,2cyclictest11431442sleep207:15:032
1137795991814,2cyclictest0-21swapper/210:22:472
113779599181,12cyclictest1393199-21apt-get11:27:362
113779599172,2cyclictest0-21swapper/212:22:532
113779599172,11cyclictest1199928-21apt-get08:12:372
1137795991714,2cyclictest1175527-21fschecks_count07:47:372
1137795991714,2cyclictest1175527-21fschecks_count07:47:372
1137795991713,3cyclictest1215064-21fschecks_count08:27:392
1137795991713,2cyclictest1457623-21aten2_r5power_p12:32:362
1137795991713,2cyclictest0-21swapper/207:27:442
1137795991712,3cyclictest1437980-21cpuspeed_turbos12:12:382
12196852162,9sleep30-21swapper/308:32:343
113779599167,7cyclictest1466152-21grep12:37:462
113779599167,7cyclictest1378924-21seq11:12:422
1137795991613,2cyclictest0-21swapper/209:56:562
1137795991613,2cyclictest0-21swapper/209:56:562
1137795991612,2cyclictest0-21swapper/209:20:432
14278432154,7sleep30-21swapper/312:02:343
12937142144,5sleep30-21swapper/309:45:013
12441712144,5sleep30-21swapper/308:55:013
13928172132,6sleep30-21swapper/311:25:023
13566002134,5sleep30-21swapper/310:47:453
13431352133,5sleep30-21swapper/310:35:013
13343612132,7sleep30-21swapper/310:27:433
12681132132,7sleep30-21swapper/309:17:483
12343202132,6sleep30-21swapper/308:45:023
12136152133,7sleep30-21swapper/308:22:503
11559242133,6sleep30-21swapper/307:27:453
11419472132,7sleep30-21swapper/307:12:483
14644952122,6sleep30-21swapper/312:37:453
14531852122,6sleep30-21swapper/312:27:423
14169172122,6sleep30-21swapper/311:47:533
14118172122,6sleep30-21swapper/311:42:473
14035932122,6sleep30-21swapper/311:37:423
13985132122,6sleep30-21swapper/311:32:403
13828352122,6sleep30-21swapper/311:14:343
13734222122,6sleep30-21swapper/311:07:243
13681092122,5sleep30-21swapper/311:00:013
13389662122,6sleep30-21swapper/310:32:413
13309572123,6sleep30-21swapper/310:22:453
13274972122,6sleep30-21swapper/310:17:483
13228532122,6sleep30-21swapper/310:12:543
13183892122,6sleep30-21swapper/310:10:013
12588812122,6sleep30-21swapper/309:08:333
12489942122,6sleep30-21swapper/308:58:343
12184842122,6sleep30-21swapper/308:27:483
12048962122,6sleep30-21swapper/308:17:373
12035192122,6sleep30-21swapper/308:12:473
11900372121,7sleep30-21swapper/308:02:363
11740722122,6sleep30-21swapper/307:42:543
11499902123,5sleep30-21swapper/307:20:023
14588752112,5sleep30-21swapper/312:32:433
14475302112,5sleep30-21swapper/312:21:083
14422692111,6sleep30-21swapper/312:15:023
14377592112,5sleep30-21swapper/312:12:373
14327162112,5sleep30-21swapper/312:06:353
14228752113,5sleep30-21swapper/311:57:123
14228752113,5sleep30-21swapper/311:57:123
14177932112,6sleep30-21swapper/311:50:343
13881382112,6sleep30-21swapper/311:21:003
13831082112,5sleep30-21swapper/311:15:023
13634002112,6sleep30-21swapper/310:55:563
13486892112,6sleep30-21swapper/310:42:243
13087642112,6sleep30-21swapper/310:00:263
13039192112,5sleep30-21swapper/309:57:093
13039192112,5sleep30-21swapper/309:57:093
12992632112,6sleep30-21swapper/309:52:403
12839782112,5sleep30-21swapper/309:35:023
12793492112,6sleep30-21swapper/309:32:373
12746552112,6sleep30-21swapper/309:27:393
12689182111,5sleep30-21swapper/309:20:013
12591572111,6sleep30-21swapper/309:10:023
12503392112,5sleep30-21swapper/309:02:433
12296162112,6sleep30-21swapper/308:40:283
12243972112,5sleep30-21swapper/308:35:023
11849212112,6sleep30-21swapper/307:55:343
11799852112,6sleep30-21swapper/307:50:323
11649282112,6sleep30-21swapper/307:36:443
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional