You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-18 - 19:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Thu Sep 18, 2025 12:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
130382722020,3sleep3662-21kworker/3:2+events09:30:553
12419022730,3chrt662-21kworker/3:2-events08:30:483
12111112560,4sleep10-21swapper/108:00:411
1159372995546,6cyclictest1180931-21latency_hist07:30:011
1159368995444,6cyclictest2583070-21systemd-journal09:55:000
1159372995039,8cyclictest1393657-21ntpq11:00:531
1159368995040,7cyclictest1419398-21apt-get11:30:000
1159372994940,6cyclictest1225587-21latency_hist08:15:001
1159372994839,7cyclictest2583070-21systemd-journal07:39:151
1159368994842,4cyclictest1469431-21apt-get12:20:440
1159372994738,6cyclictest1459241-21latency_hist12:10:011
1159372994538,5cyclictest1245418-21latency_hist08:35:011
1159372994534,8cyclictest2583070-21systemd-journal11:11:481
1159368994537,6cyclictest1210825-21latency_hist08:00:010
1159368994534,8cyclictest1384349-21apt-get10:55:010
1159372994435,7cyclictest1335112-21/usr/sbin/munin10:06:021
1159368994438,4cyclictest1275340-21kworker/u8:1+events_unbound09:15:000
1159372994337,4cyclictest1106639-21kworker/u8:0+flush-8:007:50:461
1159368994331,8cyclictest2583070-21systemd-journal08:50:000
1159368994232,7cyclictest2583070-21systemd-journal07:45:000
1159372994136,3cyclictest1154260-21kworker/u8:2+flush-8:008:20:471
1159372994132,6cyclictest1324924-21latency_hist09:55:011
1159372994029,8cyclictest2583070-21systemd-journal12:32:361
1159372993935,3cyclictest1294251-21kworker/u8:2+flush-8:010:46:021
1159372993934,3cyclictest1250603-21kworker/u8:0+flush-8:009:00:431
1159372993931,5cyclictest1413584-21ntpq11:20:551
1159372993931,5cyclictest1200057-21ntpq07:45:531
115902523927,6sleep10-21swapper/107:08:051
1159374993829,7cyclictest2583070-21systemd-journal10:24:152
1159372993834,3cyclictest1106639-21kworker/u8:0+events_unbound08:06:001
1159372993832,4cyclictest1454307-21latency_hist12:05:011
1159372993831,5cyclictest1154260-21kworker/u8:2+flush-8:007:10:451
1159372993830,6cyclictest1398674-21ntpq11:05:551
1159372993829,7cyclictest1275340-21kworker/u8:1+flush-8:009:16:071
1159372993829,7cyclictest1275340-21kworker/u8:1+flush-8:009:16:071
1159372993828,7cyclictest1423723-21ntpq11:30:551
1159372993730,5cyclictest2583070-21systemd-journal11:20:001
1159372993631,4cyclictest1294251-21kworker/u8:2+flush-8:010:35:461
1159372993630,4cyclictest1439781-21apt-get11:50:411
1159372993630,4cyclictest1439781-21apt-get11:50:411
1159368993630,4cyclictest1434077-21kworker/u8:3+flush-8:012:15:480
1159368993627,7cyclictest1186415-21diskstats07:35:450
1159372993531,3cyclictest1106639-21kworker/u8:0+flush-8:007:56:051
1159372993530,3cyclictest1225838-21kworker/u8:1+flush-8:008:25:461
1159368993525,7cyclictest1448435-21ntpq11:55:550
1159374993429,3cyclictest1235533-21turbostat.cron08:25:002
1159368993425,6cyclictest1279787-21sshd09:06:310
1159372993328,4cyclictest1384811-21kworker/u8:3+events_unbound10:55:561
1159372993327,4cyclictest2583070-21systemd-journal11:28:191
1159372993327,4cyclictest1275413-21apt-get09:05:421
1159372993327,4cyclictest1250603-21kworker/u8:0+events_unbound09:30:481
1159372993325,6cyclictest1225889-21apt-get08:15:411
1159368993325,6cyclictest1219766-21ntpq08:05:540
1159368993319,2cyclictest161rcu_preempt07:20:530
1159372993226,4cyclictest1191147-21apt-get07:40:391
1159372993225,5cyclictest1249500-21ntpq08:35:561
1159372993225,5cyclictest1225838-21kworker/u8:1+events_unbound08:45:561
1159372993222,8cyclictest1313935-21ntpq09:40:591
11591202324,9sleep20-21swapper/207:09:182
1159372993126,3cyclictest1446862-21kthreadcore11:55:521
1159372993124,5cyclictest1154260-21kworker/u8:2+events_unbound07:15:451
1159372993122,7cyclictest1369809-21/usr/sbin/munin10:40:541
115936899314,15cyclictest1241226-21if_enp4s0.308:30:460
1159368993126,3cyclictest1200556-21unixbench_multi07:46:000
1159372993025,3cyclictest1465035-21irqcore12:15:471
1159372993023,5cyclictest1250603-21kworker/u8:0+events_unbound09:35:591
115936899305,14cyclictest663-21runrttasks08:35:420
1159368993022,6cyclictest1265177-21latency_hist08:55:010
1159368993020,8cyclictest1453412-21ntpq12:00:530
1159368993019,3cyclictest421rcuog/210:45:430
1159374992924,3cyclictest1488912-21apt-get12:40:002
1159372992925,3cyclictest1470020-21irqcore12:20:521
1159372992925,3cyclictest1314749-21kworker/u8:1+flush-8:012:36:071
1159372992924,3cyclictest1267646-21kthreadcore08:55:501
115936899299,5cyclictest1460559-21cat12:10:480
115936899298,6cyclictest1269719-21grep08:56:010
115936899298,12cyclictest1350359-21perl10:20:520
115936899295,7cyclictest1255322-21idleruntime-cro08:45:000
115936899294,4cyclictest1309002-21ntp_states09:35:590
115936899292,26cyclictest3973087-21kworker/0:2+events10:25:500
1159368992919,8cyclictest1433611-21ntpq11:40:550
1159368992914,2cyclictest0-21swapper/011:00:420
115936899291,16cyclictest1166753-21hddtemp_smartct07:15:450
1159368992910,11cyclictest0-21swapper/010:16:010
11950872280,6sleep3662-21kworker/3:2+events07:40:563
1159374992823,3cyclictest1429157-21unixbench_multi11:35:592
1159372992824,3cyclictest1106639-21kworker/u8:0+flush-8:007:30:461
1159372992823,4cyclictest1359242-21kworker/u8:3+events_unbound10:30:441
1159372992822,4cyclictest1344988-21kworker/u8:0+events_unbound10:25:471
1159372992821,5cyclictest2583070-21systemd-journal10:13:151
115936899287,14cyclictest1403649-21ntpq11:10:570
115936899287,11cyclictest1477440-21kthreadcore12:25:520
115936899283,10cyclictest1479609-21uniq12:30:440
1159368992822,4cyclictest1409461-21sendmail11:20:000
1159368992822,4cyclictest1294251-21kworker/u8:2+flush-8:009:57:210
1159368992821,5cyclictest1299883-21latency_hist09:30:010
1159368992813,2cyclictest0-21swapper/011:46:010
1159368992813,2cyclictest0-21swapper/009:45:470
1159374992724,2cyclictest1429920-21aten2_r5power_c11:40:432
1159374992719,5cyclictest2583070-21systemd-journal08:45:012
1159372992723,3cyclictest389-21dbus-daemon11:45:001
1159372992723,3cyclictest1314749-21kworker/u8:1+flush-8:009:56:011
1159372992720,5cyclictest1434077-21kworker/u8:3+flush-8:012:10:541
1159372992718,7cyclictest1424889-21apt-get11:35:401
115936899272,7cyclictest1369820-21find10:40:410
1159368992722,3cyclictest1205890-21idleruntime-cro07:55:000
1159368992722,3cyclictest0-21swapper/010:15:010
1159368992720,5cyclictest1428713-21ntpq11:35:550
1159368992719,6cyclictest1444391-21latency_hist11:55:000
1159368992719,6cyclictest1444391-21latency_hist11:55:000
1159368992718,7cyclictest1294084-21ntpq09:20:570
115936899271,17cyclictest1419904-21/usr/sbin/munin11:31:000
115936899271,13cyclictest1337244-21cut10:05:530
115936899270,17cyclictest1459243-21munin-run12:10:000
115936899270,11cyclictest1221901-21grep08:10:490
1159374992623,2cyclictest1479527-21cpuspeed_turbos12:30:422
1159374992623,2cyclictest1196689-21fschecks_time07:45:452
1159374992622,2cyclictest1320276-21aten2_r5power_c09:50:432
1159374992621,3cyclictest1220321-21unixbench_singl08:06:022
1159374992619,5cyclictest2583070-21systemd-journal11:16:202
1159372992621,3cyclictest1314749-21kworker/u8:1+events_unbound11:45:501
1159368992620,4cyclictest21855-21cron11:25:000
1159368992617,3cyclictest161rcu_preempt08:15:460
115936899261,12cyclictest1181331-21aten2_r5power_v07:30:410
115936899260,15cyclictest1211513-21hddtemp_smartct08:00:460
1159372992521,3cyclictest1344988-21kworker/u8:0+events_unbound10:15:441
1159372992519,4cyclictest1250603-21kworker/u8:0+flush-8:009:10:501
115936899252,9cyclictest1239566-21ntpq08:25:510
115936899252,15cyclictest1394825-21find11:05:400
115936899252,13cyclictest1487033-21cut12:35:570
115936899252,13cyclictest1367008-21cat10:35:500
1159368992518,2cyclictest161rcu_preempt07:25:040
115936899251,6cyclictest1384807-21apt-get10:55:030
115936899251,12cyclictest1362197-21kthreadcore10:30:500
115936899250,15cyclictest1300433-21diskstats09:30:480
115936899250,10cyclictest1286036-21cut09:15:550
115936899250,10cyclictest1286036-21cut09:15:550
1159374992421,2cyclictest1211485-21fschecks_time08:00:462
1159374992420,2cyclictest1305227-21aten2_r5power_p09:35:442
1159374992420,2cyclictest1186278-21aten2_r5power_v07:35:432
1159374992420,2cyclictest1176544-21cpuspeed_turbos07:25:502
1159372992422,1cyclictest0-21swapper/110:01:001
1159372992420,3cyclictest1275340-21kworker/u8:1+flush-8:009:25:461
1159372992420,2cyclictest0-21swapper/112:25:581
1159372992419,4cyclictest1154260-21kworker/u8:2+events_unbound07:20:441
1159372992417,5cyclictest1349988-21apt-get10:20:441
115936899249,2cyclictest0-21swapper/007:10:500
115936899242,12cyclictest1312235-21kthreadcore09:40:520
1159368992414,3cyclictest161rcu_preempt09:00:470
115936899241,10cyclictest1331676-21cat10:00:500
115936899240,11cyclictest1159370-21cat07:10:010
14623912230,2sleep3481ktimers/312:10:513
115937499237,2cyclictest11821252kernelversion07:30:472
1159374992318,4cyclictest0-21swapper/209:00:002
1159374992317,4cyclictest1329886-21sendmail10:00:002
1159374992313,8cyclictest2583070-21systemd-journal10:20:002
1159372992319,3cyclictest1250603-21kworker/u8:0+flush-8:009:23:451
1159372992318,3cyclictest1315951-21kthreadcore09:45:471
1159372992318,3cyclictest0-21swapper/108:55:001
1159372992317,4cyclictest1379409-21kworker/u8:0+flush-8:010:50:491
115936899232,6cyclictest13-21ksoftirqd/008:20:460
115937499227,3cyclictest11683382kthreadcore07:15:502
1159374992219,2cyclictest1465372-21kthreadcore12:15:482
1159374992219,2cyclictest1330492-21fschecks_count10:00:462
1159374992218,2cyclictest0-21swapper/207:15:002
1159374992217,3cyclictest1296257-21kthreadcore09:25:502
1159374992216,4cyclictest1384811-21kworker/u8:3+flush-8:011:25:502
1159374992118,2cyclictest1310470-21fschecks_time09:40:472
1159374992117,3cyclictest1449341-21cron12:00:002
1159374992117,2cyclictest1240792-21aten2_r5power_c08:30:422
1159374992117,2cyclictest0-21swapper/209:50:012
1159374992113,6cyclictest1404441-21cron11:15:002
1159372992114,5cyclictest1250675-21apt-get08:40:411
11817532200,2chrt451irq_work/307:30:453
115937499202,13cyclictest1225889-21apt-get08:15:422
1159374992017,2cyclictest1171397-21aten2_r5power_e07:20:422
1159374992016,2cyclictest1380003-21fschecks_time10:50:442
1159374992016,2cyclictest0-21swapper/209:05:002
1159374992015,3cyclictest1335197-21aten2_r5power_e10:05:452
1159374992015,3cyclictest0-21swapper/207:45:012
1159374991915,2cyclictest1474420-21aten2_r5power_e12:25:442
1159374991915,2cyclictest1304476-21unixbench_singl09:31:042
1159374991915,2cyclictest1264815-21unixbench_singl08:51:012
1159374991910,7cyclictest1355665-21sed10:25:482
135455321812,4sleep30-21swapper/310:24:213
13301992184,10sleep30-21swapper/310:00:423
12052562189,5sleep30-21swapper/307:51:013
115937499182,15cyclictest0-21swapper/209:16:062
115937499182,15cyclictest0-21swapper/209:16:062
115937499182,11cyclictest1469431-21apt-get12:20:462
1159374991814,2cyclictest0-21swapper/210:55:582
1159374991814,2cyclictest0-21swapper/210:50:012
1159374991813,3cyclictest1202594-21kthreadcore07:50:502
1159374991813,3cyclictest0-21swapper/211:30:502
11542672182,8sleep30-21swapper/307:05:023
14804192170,2chrt451irq_work/312:30:483
115937499178,6cyclictest1399485-21cron11:10:002
1159374991713,2cyclictest0-21swapper/212:00:452
1159374991713,2cyclictest0-21swapper/211:50:492
1159374991713,2cyclictest0-21swapper/211:50:492
1159374991713,2cyclictest0-21swapper/210:36:002
1159374991712,3cyclictest0-21swapper/211:46:012
1159374991712,3cyclictest0-21swapper/210:10:582
1159374991712,3cyclictest0-21swapper/209:21:032
1159374991711,4cyclictest0-21swapper/208:10:522
115937499171,12cyclictest1209906-21ntpq07:55:582
115937499168,6cyclictest1240450-21cron08:30:002
1159374991612,2cyclictest0-21swapper/212:09:522
1159374991612,2cyclictest0-21swapper/210:31:002
1159374991612,2cyclictest0-21swapper/208:41:012
1159374991611,3cyclictest0-21swapper/209:10:482
1159374991611,3cyclictest0-21swapper/209:05:562
1159374991611,3cyclictest0-21swapper/208:35:522
115937499161,11cyclictest1410981-21kthreadcore11:20:502
115937499156,7cyclictest1390218-21tune2fs11:00:442
1159374991511,3cyclictest0-21swapper/212:10:572
1159374991511,2cyclictest0-21swapper/210:41:012
14109772132,7sleep30-21swapper/311:20:513
14084302132,7sleep30-21swapper/311:15:533
12261272132,7sleep30-21swapper/308:15:443
12165202132,7sleep30-21swapper/308:05:493
14691422122,5sleep30-21swapper/312:20:013
14685462122,6sleep30-21swapper/312:16:003
14501402122,6sleep30-21swapper/312:00:473
14485182123,5sleep30-21swapper/311:55:583
14295752122,5sleep30-21swapper/311:40:013
14196912121,6sleep30-21swapper/311:30:023
13998012122,6sleep30-21swapper/311:10:443
13907672122,6sleep30-21swapper/311:00:483
13747622122,6sleep30-21swapper/310:45:443
13638432122,6sleep30-21swapper/310:30:593
13546412122,5sleep30-21swapper/310:25:013
13460992122,6sleep30-21swapper/310:15:503
13356442122,6sleep30-21swapper/310:05:513
13280522122,6sleep30-21swapper/309:55:523
13211632122,6sleep30-21swapper/309:50:503
13048552122,5sleep30-21swapper/309:35:013
12961332122,5sleep30-21swapper/309:25:513
12897192122,6sleep30-21swapper/309:17:003
12897192122,6sleep30-21swapper/309:17:003
12842572122,6sleep30-21swapper/309:10:583
12609242122,6sleep30-21swapper/308:50:473
12553582122,5sleep30-21swapper/308:45:023
12308452122,6sleep30-21swapper/308:20:423
12100182122,6sleep30-21swapper/307:56:023
11984902122,6sleep30-21swapper/307:45:513
11633772123,6sleep30-21swapper/307:10:563
14888242112,6sleep30-21swapper/312:39:033
14787142113,5sleep30-21swapper/312:26:363
14590192112,6sleep30-21swapper/312:07:213
14397472112,6sleep30-21swapper/311:50:363
14397472112,6sleep30-21swapper/311:50:363
14393532112,5sleep30-21swapper/311:48:323
14289482112,5sleep30-21swapper/311:36:003
14147162112,5sleep30-21swapper/311:25:443
13945652112,5sleep30-21swapper/311:05:023
13848912112,5sleep30-21swapper/310:55:483
13796932112,5sleep30-21swapper/310:50:413
13697622112,6sleep30-21swapper/310:40:063
13647982112,6sleep30-21swapper/310:35:163
13425072112,5sleep30-21swapper/310:10:493
13150722112,5sleep30-21swapper/309:45:413
13144872112,6sleep30-21swapper/309:42:043
12918882113,5sleep30-21swapper/309:20:533
12753692112,6sleep30-21swapper/309:05:283
12704242112,5sleep30-21swapper/309:00:393
12654502112,5sleep30-21swapper/308:55:413
12511522112,5sleep30-21swapper/308:40:483
12502732112,6sleep30-21swapper/308:39:083
12395682112,5sleep30-21swapper/308:25:523
12213492112,5sleep30-21swapper/308:10:473
12111402112,5sleep30-21swapper/308:00:443
11861232112,6sleep30-21swapper/307:35:283
11767022112,5sleep30-21swapper/307:25:533
11741612112,5sleep30-21swapper/307:20:513
11653512112,5sleep30-21swapper/307:15:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional