You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-20 - 20:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Fri Mar 20, 2026 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14898812980,2sleep375-21kworker/3:1-events11:21:423
12658592670,19sleep375-21kworker/3:1+events07:36:393
13067562640,2sleep3481ktimers/308:16:403
14114782630,5chrt0-21swapper/310:01:413
13105602560,4chrt0-21swapper/108:21:391
1237149995646,7cyclictest1476999-21latency_hist11:10:010
1237149995246,4cyclictest1298722-21latency_hist08:10:010
1237152994437,5cyclictest1254172-21kworker/u8:2+flush-8:008:06:471
1237152994336,5cyclictest1372800-21kworker/u8:3+flush-8:010:46:461
1237149994336,5cyclictest1253943-21latency_hist07:25:000
1237149994335,6cyclictest1353348-21/usr/sbin/munin09:06:480
1237155994121,7cyclictest1451749-21unixbench_multi10:41:502
1237152994134,5cyclictest1289111-21kworker/u8:1+flush-8:009:06:351
1237152994133,6cyclictest1302777-21ntpq08:11:421
1237152994031,7cyclictest1401905-21ntpq09:51:451
1237152994030,8cyclictest1530592-21ntpq12:01:431
1237149993930,7cyclictest1377731-21latency_hist09:30:010
12322072397,3sleep00-21swapper/007:05:390
12322072397,3sleep00-21swapper/007:05:390
1237155993821,4cyclictest1304257-21fschecks_count08:16:342
1237149993813,23cyclictest1362919-21kworker/0:1+events11:56:420
1237155993721,4cyclictest1373426-21fschecks_count09:26:342
1237152993733,3cyclictest1231952-21kworker/u8:0+flush-8:009:36:351
1237152993732,3cyclictest1275095-21irqrtprio07:46:391
1237152993730,5cyclictest1382916-21kworker/u8:2+flush-8:010:06:491
1237152993728,7cyclictest280-21systemd-journal11:35:541
123714999371,34cyclictest1392652-21munin-run09:45:000
1237149993711,24cyclictest1362919-21kworker/0:1+events10:46:420
13361612360,4sleep1281rcuc/108:46:401
1237152993630,4cyclictest1254172-21kworker/u8:2+flush-8:008:01:361
1237149993627,7cyclictest1452102-21apt-get10:45:010
1237149993626,8cyclictest1482548-21diskstats11:16:340
1237155993519,4cyclictest1254525-21cpuspeed_turbos07:26:322
123714999358,22cyclictest1268980-21kworker/0:2+events08:00:540
1237149993527,6cyclictest1308571-21latency_hist08:20:010
1237155993421,3cyclictest1319086-21fschecks_count08:31:342
1237155993420,3cyclictest1497398-21fschecks_count11:31:352
1237155993420,3cyclictest1497398-21fschecks_count11:31:352
1237152993429,3cyclictest1254172-21kworker/u8:2+flush-8:007:30:311
1237152993427,5cyclictest1412802-21kworker/u8:1+flush-8:010:31:341
123714999349,23cyclictest1362919-21kworker/0:1+events09:56:340
1237149993425,7cyclictest1274350-21/usr/sbin/munin07:46:420
123714999342,30cyclictest1362919-21kworker/0:1+events11:11:300
15322942330,3sleep3451irq_work/312:06:363
1237155993320,3cyclictest1542001-21aten2_r5power_e12:16:322
1237152993328,3cyclictest1552560-21irqrtprio12:26:381
1237152993325,6cyclictest1382916-21kworker/u8:2+flush-8:010:56:351
1237152993324,7cyclictest1427801-21apt-get10:21:311
1237152993324,6cyclictest280-21systemd-journal12:00:001
1237149993326,5cyclictest1497050-40mandb11:30:520
1237149993326,5cyclictest1497050-40mandb11:30:520
1237149993324,7cyclictest1521523-21latency_hist11:55:020
1237149993323,8cyclictest1352158-21ntpq09:01:430
123714999331,26cyclictest1268980-21kworker/0:2+events07:41:320
1237149993311,16cyclictest1268980-21kworker/0:2+events08:51:300
1237149993311,16cyclictest1236876-21kworker/0:0+events07:10:020
1237149993311,16cyclictest1236876-21kworker/0:0+events07:10:020
12368012334,9sleep10-21swapper/107:09:491
12368012334,9sleep10-21swapper/107:09:491
1237152993227,4cyclictest1231952-21kworker/u8:0+flush-8:009:31:351
1237152993224,6cyclictest1372800-21kworker/u8:3+flush-8:010:41:491
123714999327,20cyclictest1268980-21kworker/0:2+events08:45:000
1237149993227,3cyclictest0-21swapper/009:15:000
1237149993224,5cyclictest1551471-21latency_hist12:25:010
123714999322,28cyclictest1362919-21kworker/0:1+events11:46:340
123714999322,28cyclictest1362919-21kworker/0:1+events11:21:380
123714999322,28cyclictest1362919-21kworker/0:1+events09:46:320
123714999321,30cyclictest1362919-21kworker/0:1+events10:38:100
13153332310,3sleep3471rcuc/308:26:393
1237155993127,3cyclictest1328328-21idleruntime-cro08:40:002
1237155993127,2cyclictest0-21swapper/208:55:002
1237155993121,7cyclictest1442869-21fschecks_count10:36:342
1237152993127,2cyclictest1393287-21aten2_r5power_p09:46:331
1237152993126,3cyclictest1231952-21kworker/u8:0+events_unbound07:51:501
1237152993125,4cyclictest1289111-21kworker/u8:1+flush-8:009:21:341
1237152993125,4cyclictest1254172-21kworker/u8:2+flush-8:007:41:351
1237152993124,5cyclictest1372800-21kworker/u8:3+flush-8:012:36:481
123714999319,16cyclictest1362919-21kworker/0:1+events11:01:300
123714999319,16cyclictest1362919-21kworker/0:1+events10:56:300
123714999317,22cyclictest1236876-21kworker/0:0+events07:16:440
123714999317,22cyclictest1236876-21kworker/0:0+events07:16:440
123714999316,23cyclictest1362919-21kworker/0:1+events10:51:440
123714999314,22cyclictest1268980-21kworker/0:2+events08:45:260
1237149993125,4cyclictest1428019-21cpuspeed_turbos10:21:340
123714999312,28cyclictest1521544-21kworker/0:2+events12:10:000
123714999312,27cyclictest1362919-21kworker/0:1+events10:26:380
123714999312,27cyclictest1362919-21kworker/0:1+events09:51:300
1237149993122,7cyclictest1289111-21kworker/u8:1+flush-8:009:31:480
123714999312,27cyclictest1249030-21kworker/0:1+events07:36:460
1237149993121,8cyclictest1322537-21ntpq08:31:420
1237149993120,9cyclictest1312695-21ntpq08:21:430
123714999311,28cyclictest1362919-21kworker/0:1+events10:31:300
123714999311,28cyclictest1249030-21kworker/0:1+events07:26:300
1237155993023,5cyclictest280-21systemd-journal12:26:272
1237155993021,7cyclictest1428065-21fschecks_count10:21:352
1237155993021,7cyclictest1314147-21fschecks_count08:26:342
1237152993025,3cyclictest1532009-21apt-get12:06:321
1237152993025,3cyclictest1494347-21kthreadcore11:26:401
1237152993025,3cyclictest1237142-21cyclictest08:41:481
1237152993025,3cyclictest1237142-21cyclictest08:41:481
123714999308,16cyclictest1362919-21kworker/0:1+events10:06:320
123714999305,23cyclictest1521544-21kworker/0:2+events12:01:420
1237149993027,2cyclictest1427433-21sendmail10:20:000
123714999302,27cyclictest1362919-21kworker/0:1+events11:39:240
123714999302,26cyclictest1268980-21kworker/0:2+events08:26:320
123714999301,27cyclictest1506742-21kworker/0:0+events11:41:360
123714999301,27cyclictest1268980-21kworker/0:2+events08:41:440
123714999300,28cyclictest1521544-21kworker/0:2+events12:26:380
1237155992925,2cyclictest1532323-21fschecks_time12:06:362
1237155992925,2cyclictest1436935-21unixbench_multi10:26:492
1237155992924,3cyclictest1537073-21aten2_r5power_v12:11:302
1237155992921,6cyclictest280-21systemd-journal11:20:542
1237155992919,8cyclictest1358626-21fschecks_count09:11:362
1237152992924,4cyclictest1231952-21kworker/u8:0+flush-8:008:36:361
1237152992924,3cyclictest1231952-21kworker/u8:0+flush-8:007:36:471
1237152992923,4cyclictest1412802-21kworker/u8:1+flush-8:011:51:371
1237152992923,4cyclictest1241820-21unixbench_multi07:11:511
1237152992923,4cyclictest1241820-21unixbench_multi07:11:511
123714999298,19cyclictest1268980-21kworker/0:2+events07:57:160
123714999298,17cyclictest1362919-21kworker/0:1+events09:35:260
123714999295,18cyclictest1362919-21kworker/0:1+events11:30:000
123714999294,23cyclictest1268980-21kworker/0:2+events09:16:460
1237155992822,4cyclictest1294464-21fschecks_time08:06:352
1237155992819,7cyclictest1363560-21fschecks_time09:16:352
1237155992819,7cyclictest1259559-21fschecks_count07:31:342
1237152992824,3cyclictest1231952-21kworker/u8:0+flush-8:007:26:341
1237152992820,5cyclictest1237358-21kworker/u8:3+flush-8:007:21:381
123714999289,17cyclictest1521544-21kworker/0:2+events12:31:400
1237155992723,3cyclictest1408262-21fschecks_count10:01:352
1237155992721,4cyclictest1561969-21fschecks_count12:36:342
1237155992721,4cyclictest1457874-21fschecks_count10:51:342
1237155992719,6cyclictest1333992-21fschecks_time08:46:342
1237155992719,6cyclictest1303280-21unixbench_multi08:11:502
1237155992718,7cyclictest1502422-21fschecks_time11:36:352
1237152992722,3cyclictest1382916-21kworker/u8:2+flush-8:010:35:251
1237152992721,4cyclictest1372800-21kworker/u8:3+events_unbound11:41:391
1237152992719,6cyclictest1541917-21/usr/sbin/munin12:16:421
1237152992718,7cyclictest1476164-21ntpq11:06:421
1237152992718,6cyclictest280-21systemd-journal10:05:001
123714999279,17cyclictest1249030-21kworker/0:1+events07:31:460
123714999278,18cyclictest1521544-21kworker/0:2+events12:36:380
123714999278,17cyclictest1268980-21kworker/0:2+events07:51:520
1237149992710,16cyclictest1268980-21kworker/0:2+events08:56:360
1237149992710,15cyclictest1521544-21kworker/0:2+events12:11:280
1237149992710,15cyclictest1362919-21kworker/0:1+events10:10:300
1237155992621,3cyclictest1526156-21unixbench_singl11:56:512
1237155992621,3cyclictest1481586-21unixbench_multi11:11:502
1237155992620,4cyclictest1368514-21fschecks_count09:21:342
1237155992616,8cyclictest1423133-21fschecks_count10:16:352
1237152992621,3cyclictest1382916-21kworker/u8:2+flush-8:009:56:381
1237152992621,3cyclictest1289111-21kworker/u8:1+flush-8:009:11:551
1237152992620,4cyclictest1412802-21kworker/u8:1+events_unbound10:11:541
123714999267,17cyclictest1521544-21kworker/0:2+events12:16:380
123714999267,17cyclictest1268980-21kworker/0:2+events08:11:420
123714999262,23cyclictest0-21swapper/010:01:480
1237155992521,2cyclictest1512305-21fschecks_count11:46:332
1237155992521,2cyclictest1467762-21fschecks_count11:01:342
1237155992520,3cyclictest1557059-21fschecks_time12:31:352
1237155992520,3cyclictest1462761-21cpuspeed_turbos10:56:332
1237155992517,6cyclictest1547189-21fschecks_time12:21:342
1237155992517,6cyclictest1403356-21fschecks_time09:56:342
1237155992512,9cyclictest1388011-21apt-get09:41:332
1237152992521,3cyclictest1231952-21kworker/u8:0+flush-8:008:16:471
1237152992521,2cyclictest1288847-21awk08:00:001
1237152992520,4cyclictest1382916-21kworker/u8:2+flush-8:011:01:351
1237152992520,3cyclictest1231952-21kworker/u8:0+events_unbound09:01:361
1237152992520,3cyclictest0-21swapper/110:55:001
1237152992519,4cyclictest1342800-21unixbench_multi08:51:491
1237152992518,5cyclictest1372800-21kworker/u8:3+flush-8:009:41:511
1237152992517,5cyclictest1437278-21latency_hist10:30:011
1237152992514,7cyclictest0-21swapper/112:21:311
123714999257,16cyclictest1362919-21kworker/0:1+events09:21:320
14286902240,4sleep3481ktimers/310:21:393
1237155992421,2cyclictest1328931-21cpuspeed_turbos08:41:332
1237155992421,2cyclictest1328931-21cpuspeed_turbos08:41:332
1237155992419,3cyclictest1274689-21fschecks_time07:46:352
1237155992419,3cyclictest1242151-21latency_hist07:15:012
1237155992418,4cyclictest1521173-21unixbench_multi11:51:492
1237152992420,2cyclictest876-21mta-sts-daemon11:10:191
1237152992420,2cyclictest0-21swapper/111:21:511
1237152992419,4cyclictest1323406-21systemd-run08:35:001
123714999246,16cyclictest1268980-21kworker/0:2+events08:36:340
1237155992320,2cyclictest1378320-21cpuspeed_turbos09:31:332
1237155992319,2cyclictest1507352-21fschecks_count11:41:362
1237155992317,4cyclictest1343799-21fschecks_count08:56:362
1237155992315,6cyclictest1418244-21fschecks_time10:11:382
1237152992319,3cyclictest1372800-21kworker/u8:3+flush-8:011:46:371
1237152992319,2cyclictest0-21swapper/110:16:371
1237152992318,4cyclictest1231952-21kworker/u8:0+flush-8:009:26:351
1237152992318,3cyclictest1526208-21kworker/u8:0+flush-8:012:11:351
1237152992317,4cyclictest0-21swapper/111:16:371
1237155992219,2cyclictest1495876-21needreboot11:26:422
1237155992219,2cyclictest1486860-21cron11:20:002
1237155992217,3cyclictest1472720-21fschecks_count11:06:332
1237155992217,3cyclictest0-21swapper/212:05:012
1237152992218,2cyclictest0-21swapper/111:31:381
1237152992218,2cyclictest0-21swapper/111:31:381
1237152992215,5cyclictest0-21swapper/109:16:421
1237152992214,6cyclictest1347287-21ntpq08:56:451
1237155992116,4cyclictest1267915-21needreboot07:36:422
1237155992116,3cyclictest1283524-21unixbench_multi07:51:522
14969542200,2sleep3481ktimers/311:30:003
123715599203,12cyclictest1383054-21apt-get09:36:312
1237155992015,3cyclictest1284554-21fschecks_time07:56:332
1237155992015,3cyclictest0-21swapper/209:51:392
1237155992015,3cyclictest0-21swapper/207:20:002
1237155992015,3cyclictest0-21swapper/207:20:002
1237155992014,3cyclictest0-21swapper/207:10:002
1237155992014,3cyclictest0-21swapper/207:10:002
1237152992016,2cyclictest876-21mta-sts-daemon12:31:411
1237152992015,3cyclictest0-21swapper/107:15:031
1237152992015,3cyclictest0-21swapper/107:15:031
123715599192,7cyclictest0-21swapper/207:21:432
1237155991916,2cyclictest1353009-21apt09:05:002
1237155991915,2cyclictest0-21swapper/210:10:002
1237155991915,2cyclictest0-21swapper/208:21:422
1237155991914,3cyclictest0-21swapper/207:15:022
1237152991913,4cyclictest0-21swapper/108:26:351
1237155991815,2cyclictest1397415-21unixbench_multi09:46:512
1237155991814,2cyclictest1291376-21sort08:01:402
12321922183,9sleep30-21swapper/307:05:303
12321922183,9sleep30-21swapper/307:05:303
1237155991714,2cyclictest1437994-21fschecks_time10:31:342
1237155991713,2cyclictest1353459-21aten2_r5power_p09:06:332
1237155991613,2cyclictest0-21swapper/210:45:252
1237155991611,3cyclictest0-21swapper/207:41:402
14791392145,5sleep30-21swapper/311:11:403
15157582134,5sleep30-21swapper/311:46:433
14734712132,7sleep30-21swapper/311:06:383
13916882132,7sleep30-21swapper/309:41:443
13548912132,7sleep30-21swapper/309:06:403
13482762132,6sleep30-21swapper/309:00:023
12839082132,6sleep30-21swapper/307:55:023
12589272132,6sleep30-21swapper/307:30:023
15533112122,6sleep30-21swapper/312:26:393
15471232122,6sleep30-21swapper/312:21:343
15419462122,6sleep30-21swapper/312:16:323
15368412122,6sleep30-21swapper/312:10:183
15305732123,6sleep30-21swapper/312:01:443
15215372122,5sleep30-21swapper/311:55:013
14819202122,5sleep30-21swapper/311:15:023
14663882122,6sleep30-21swapper/310:56:473
14527302122,6sleep30-21swapper/310:46:323
14471932122,6sleep30-21swapper/310:40:023
14380202122,6sleep30-21swapper/310:31:353
14327052122,6sleep30-21swapper/310:26:293
14230052122,6sleep30-21swapper/310:16:343
13990182122,6sleep30-21swapper/309:51:383
13797302122,6sleep30-21swapper/309:31:403
13579852122,5sleep30-21swapper/309:10:013
13407342122,6sleep30-21swapper/308:51:403
13184202122,5sleep30-21swapper/308:30:013
13033592122,6sleep30-21swapper/308:11:513
12985412122,6sleep30-21swapper/308:07:543
12746242122,6sleep30-21swapper/307:46:353
12734412122,6sleep30-21swapper/307:41:503
12532442122,6sleep30-21swapper/307:21:473
12445292122,6sleep30-21swapper/307:16:343
12445292122,6sleep30-21swapper/307:16:343
12418682122,6sleep30-21swapper/307:11:523
12418682122,6sleep30-21swapper/307:11:523
15615392112,5sleep30-21swapper/312:35:023
15567502112,5sleep30-21swapper/312:31:343
15209522112,5sleep30-21swapper/311:51:483
15114902112,6sleep30-21swapper/311:42:463
15062292112,5sleep30-21swapper/311:36:523
14969542112,6sleep30-21swapper/311:30:013
14969542112,6sleep30-21swapper/311:30:013
14673832112,6sleep30-21swapper/311:00:243
14620582112,6sleep30-21swapper/310:53:383
14430482111,6sleep30-21swapper/310:36:373
14219082112,5sleep30-21swapper/310:11:533
14131752112,5sleep30-21swapper/310:06:353
14029292112,6sleep30-21swapper/309:55:243
13970802112,5sleep30-21swapper/309:46:493
13827132112,5sleep30-21swapper/309:35:013
13730112112,6sleep30-21swapper/309:25:143
13681972112,5sleep30-21swapper/309:21:293
13634922111,6sleep30-21swapper/309:16:353
13473992112,5sleep30-21swapper/308:56:493
13335002112,6sleep30-21swapper/308:45:023
13285882113,5sleep30-21swapper/308:40:203
13285882113,5sleep30-21swapper/308:40:203
13236132112,5sleep30-21swapper/308:35:023
13087042112,5sleep30-21swapper/308:20:023
12891342112,6sleep30-21swapper/308:00:483
12791962112,6sleep30-21swapper/307:50:083
12548392112,5sleep30-21swapper/307:26:373
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional