You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-14 - 16:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Tue Jul 14, 2026 12:44:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
40081152650,2sleep3471rcuc/311:12:353
3764178996346,13cyclictest3769072-21latency_hist07:15:011
40057482600,4chrt0-21swapper/011:12:320
3764178995938,12cyclictest3865446-21latency_hist08:50:001
3764178995135,14cyclictest3969253-21ntpq10:32:381
3764175994840,5cyclictest3935170-21latency_hist10:00:020
3764175994839,7cyclictest3920103-21latency_hist09:45:010
3764175994637,7cyclictest281-21systemd-journal09:30:010
3764175994338,4cyclictest3899880-21kworker/u8:1+flush-8:009:52:410
3764185994235,5cyclictest281-21systemd-journal08:40:002
3764185994234,6cyclictest4059524-21gpgv12:05:012
3764178994230,10cyclictest384-21dbus-daemon12:00:011
3764175994235,6cyclictest3850405-21kworker/u8:1+flush-8:008:52:300
3764175994233,7cyclictest3985611-21diskstats10:52:280
376417899411,24cyclictest585-21runrttasks11:17:441
376417599410,39cyclictest4074577-21munin-run12:20:000
3764178994029,8cyclictest281-21systemd-journal09:57:381
3764178994021,15cyclictest281-21systemd-journal12:05:381
3764175994036,3cyclictest3850405-21kworker/u8:1+flush-8:009:07:300
3764175994032,6cyclictest3775983-21latency_hist07:20:010
3764175993933,5cyclictest3796295-21kworker/u8:2+flush-8:007:57:440
37599472390,28sleep3451irq_work/307:07:293
3764178993832,4cyclictest3791589-21grep07:37:271
3764178993830,6cyclictest281-21systemd-journal09:12:381
3764175993834,3cyclictest4084854-21kworker/u8:3+flush-8:012:37:440
376417899373,30cyclictest0-21swapper/107:27:271
376417899371,8cyclictest585-21runrttasks11:37:381
376417899364,18cyclictest0-21swapper/110:52:291
3764178993631,4cyclictest3899880-21kworker/u8:1+flush-8:010:22:441
3764178993624,10cyclictest3841212-21cut08:27:281
3764178993619,15cyclictest4085247-21/usr/sbin/munin12:32:381
3764178993619,15cyclictest4085247-21/usr/sbin/munin12:32:381
3764178993613,19cyclictest3904998-21latency_hist09:30:011
3764175993631,4cyclictest4059878-21kworker/u8:2+flush-8:012:27:430
3764175993623,10cyclictest3999000-21ntpq11:02:400
3764178993529,4cyclictest4042236-21grep11:47:351
3764178993510,15cyclictest421rcuog/210:18:591
3764175993530,4cyclictest4059878-21kworker/u8:2+flush-8:012:07:300
376417899344,15cyclictest421rcuog/208:14:581
3764178993427,5cyclictest3831069-21apt-get08:17:251
3764178993425,6cyclictest4084838-21latency_hist12:30:001
3764178993423,8cyclictest281-21systemd-journal10:45:001
3764178993422,10cyclictest0-21swapper/109:05:001
376417899341,6cyclictest38128101kthreadcore07:57:371
3764178993415,16cyclictest3839745-21ntpq08:22:401
3764175993425,6cyclictest3764007-21latency_hist07:10:000
3764175993425,6cyclictest281-21systemd-journal10:20:000
376417899336,18cyclictest421rcuog/212:13:531
3764178993328,3cyclictest3772182-21irqrtprio07:17:311
3764178993327,4cyclictest3776431-21apt-get07:22:251
3764178993325,6cyclictest3880678-21/usr/sbin/munin09:07:431
3764178993322,9cyclictest384-21dbus-daemon12:25:001
3764175993328,4cyclictest3910417-21kworker/u8:2+flush-8:009:37:300
376417899323,25cyclictest0-21swapper/112:17:271
3764178993227,3cyclictest4004731-21cron11:10:001
3764175993227,4cyclictest4029738-21kworker/u8:2+flush-8:011:37:450
3764175993227,4cyclictest3899880-21kworker/u8:1+flush-8:011:17:430
3764175993225,5cyclictest3860300-21latency_hist08:45:000
3764175993223,7cyclictest3796430-21/usr/sbin/munin07:42:380
3764175993222,8cyclictest4054464-21latency_hist12:00:010
376417899318,10cyclictest421rcuog/207:54:471
376417899314,13cyclictest30-21ksoftirqd/110:36:041
3764178993126,3cyclictest3856518-21irqrtprio08:42:311
376417899312,4cyclictest0-21swapper/109:42:261
3764178993120,9cyclictest3866229-21cut08:52:301
3764178993119,9cyclictest454-21mta-sts-daemon10:07:411
3764178993119,10cyclictest3940134-21cron10:05:001
376417899311,11cyclictest3890603-21apt-get09:17:241
3764175993125,4cyclictest3899880-21kworker/u8:1+flush-8:010:02:350
3764185993026,2cyclictest4070332-21fschecks_time12:17:312
3764185993022,6cyclictest4019644-21apt-get11:25:002
3764178993025,3cyclictest0-21swapper/109:35:001
3764178993016,12cyclictest3924221-21needreboot09:47:371
3764178993016,12cyclictest3898871-21kthreadcore09:22:351
3764178993016,12cyclictest3850882-21aten2_r5power_e08:37:281
3764175993025,4cyclictest3899880-21kworker/u8:1+flush-8:009:47:420
3764175993025,4cyclictest3796295-21kworker/u8:2+flush-8:007:47:420
3764175993020,8cyclictest281-21systemd-journal10:37:380
3764175993018,9cyclictest281-21systemd-journal12:20:030
3764185992921,6cyclictest4080336-21apt-get12:27:262
376417899293,21cyclictest0-21swapper/107:42:291
3764178992925,3cyclictest3899880-21kworker/u8:1+events_unbound09:52:311
3764178992925,2cyclictest3816791-21irqcore08:02:311
3764178992921,5cyclictest4039660-21latency_hist11:45:001
3764178992920,7cyclictest3899880-21kworker/u8:1+flush-8:012:37:301
3764178992919,8cyclictest3980407-21apt-get10:47:261
3764178992919,8cyclictest3980407-21apt-get10:47:261
3764178992916,11cyclictest4029449-21unixbench_multi11:32:461
3764178992915,12cyclictest3805662-21unixbench_multi07:47:451
376417899291,23cyclictest3945743-21apt-get10:12:251
3764175992924,4cyclictest4044585-21kworker/u8:0+flush-8:011:52:290
3764175992924,4cyclictest3899880-21kworker/u8:1+flush-8:010:07:420
3764175992924,3cyclictest3965089-21cron10:30:000
3764175992923,4cyclictest2951461-21kworker/u8:3+events_unbound07:27:390
3764175992922,5cyclictest3899880-21kworker/u8:1+flush-8:011:22:440
3764178992824,2cyclictest3998969-21needreboot11:02:391
3764178992823,3cyclictest3845908-21apt-get08:32:271
376417899282,23cyclictest3914447-21ntpq09:37:401
3764178992813,13cyclictest3788638-21kthreadcore07:32:371
376417899281,24cyclictest582-21snmpd11:27:411
376417899281,21cyclictest582-21snmpd12:02:421
3764178992812,14cyclictest3870821-21acpi08:57:251
3764175992824,2cyclictest3849600-21anacron08:32:390
3764175992824,2cyclictest0-21swapper/007:25:010
3764175992823,4cyclictest4044585-21kworker/u8:0+flush-8:012:32:430
3764175992823,4cyclictest4044585-21kworker/u8:0+flush-8:012:32:430
3764175992823,4cyclictest4014674-21kworker/u8:3+events_unbound11:27:460
3764175992823,3cyclictest3905654-21cpuspeed_turbos09:32:300
3764175992823,3cyclictest2951461-21kworker/u8:3+events_unbound07:40:010
3764175992821,5cyclictest4059179-21unixbench_singl12:02:460
3764175992819,7cyclictest3764032-21kworker/u8:0+flush-8:008:27:370
38134962270,2sleep3584-21kworker/3:2+events07:57:363
376417899273,20cyclictest4015089-21apt-get11:22:261
3764178992723,3cyclictest3965120-21systemd-run10:30:001
3764178992717,7cyclictest0-21swapper/111:00:011
3764175992723,3cyclictest3796295-21kworker/u8:2+flush-8:008:22:320
3764175992722,4cyclictest4029738-21kworker/u8:2+flush-8:011:42:310
3764175992722,4cyclictest3899880-21kworker/u8:1+flush-8:010:32:340
3764175992722,4cyclictest3885227-21kworker/u8:3+flush-8:009:17:410
3764175992722,3cyclictest4000805-21irqcore11:07:330
3764175992721,4cyclictest451-21mta-sts-daemon10:22:450
3764175992720,5cyclictest3764032-21kworker/u8:0+events_unbound08:17:410
3764185992622,3cyclictest3915634-21aten2_r5power_c09:42:272
3764185992622,2cyclictest3885731-21aten2_r5power_p09:12:282
3764185992622,2cyclictest3821487-21fschecks_time08:07:292
3764185992621,4cyclictest4059878-21kworker/u8:2+flush-8:012:22:312
3764185992621,4cyclictest3875307-21idleruntime-cro09:00:002
3764185992621,3cyclictest4009431-21unixbench_multi11:12:452
3764185992620,4cyclictest3860589-21latency_hist08:45:002
376417899262,3cyclictest0-21swapper/108:07:411
376417899262,18cyclictest451-21mta-sts-daemon11:12:251
376417899261,5cyclictest3764179-21latency_hist07:10:001
3764175992622,3cyclictest4029738-21kworker/u8:2+flush-8:011:47:430
3764175992621,4cyclictest3930459-21kworker/u8:3+flush-8:010:12:340
3764175992621,3cyclictest0-21swapper/008:05:000
3764175992620,4cyclictest3850405-21kworker/u8:1+events_unbound08:37:300
376417599261,24cyclictest3754291-21kworker/0:0+events09:02:350
3764185992522,2cyclictest3965752-21cpuspeed_turbos10:32:282
3764185992521,2cyclictest4060305-21cpuspeed_turbos12:07:292
376417899252,2cyclictest0-21swapper/111:52:251
3764175992521,3cyclictest3980205-21kworker/u8:2+flush-8:010:57:310
3764175992521,3cyclictest2951461-21kworker/u8:3+flush-8:007:32:480
3764175992520,3cyclictest3874415-21wc08:57:370
3764175992518,5cyclictest3769397-21kthreadcore07:15:010
3764185992421,2cyclictest4090318-21aten2_r5power_p12:37:282
3764185992421,2cyclictest4040123-21aten2_r5power_c11:47:282
3764185992420,2cyclictest3831183-21aten2_r5power_v08:17:272
3764185992420,2cyclictest0-21swapper/211:55:002
3764185992420,2cyclictest0-21swapper/211:00:002
3764185992419,4cyclictest3939914-21unixbench_singl10:02:472
3764175992419,3cyclictest3764032-21kworker/u8:0+events_unbound07:52:290
3764175992418,4cyclictest3980205-21kworker/u8:2+flush-8:010:50:000
3764175992418,4cyclictest3980205-21kworker/u8:2+flush-8:010:50:000
3764185992319,3cyclictest3909730-21unixbench_singl09:32:492
3764175992318,4cyclictest3979988-21systemd-run10:45:000
3764175992317,4cyclictest3796295-21kworker/u8:2+flush-8:009:22:420
3764175992317,4cyclictest3764032-21kworker/u8:0+events_unbound08:10:000
3764185992219,2cyclictest3805709-21unixbench_singl07:47:452
3764185992218,3cyclictest3806686-21fschecks_time07:52:312
3764185992218,3cyclictest0-21swapper/211:45:002
3764185992218,2cyclictest4000327-21aten2_r5power_v11:07:282
3764185992217,4cyclictest3819918-21needreboot08:02:362
3764185992217,4cyclictest19289-21cron07:35:002
3764185992217,3cyclictest4054234-21unixbench_singl11:57:472
3764185992217,3cyclictest0-21swapper/207:30:002
3764185992215,5cyclictest4029107-21grep11:32:432
3764175992218,3cyclictest3764032-21kworker/u8:0+flush-8:008:12:430
3764175992217,4cyclictest3899880-21kworker/u8:1+events_unbound12:13:020
3764175992216,5cyclictest3899880-21kworker/u8:1+flush-8:011:32:410
3764175992216,4cyclictest3796295-21kworker/u8:2+events_unbound09:15:020
376418599211,9cyclictest421rcuog/211:02:292
3764185992117,2cyclictest3980519-21aten2_r5power_v10:47:292
3764185992117,2cyclictest3980519-21aten2_r5power_v10:47:282
3764185992116,4cyclictest3885205-21idleruntime-cro09:10:002
3764185992112,7cyclictest3781193-21cat07:25:012
3764175992117,2cyclictest0-21swapper/008:47:270
3764185992016,3cyclictest3940923-21aten2_r5power_v10:07:272
3764185992016,2cyclictest0-21swapper/207:15:012
3764185991916,2cyclictest3796009-21idleruntime-cro07:40:002
3764185991915,3cyclictest3826166-21aten2_r5power_e08:12:272
3764185991915,2cyclictest0-21swapper/207:20:012
3764185991914,3cyclictest3800696-21unixbench_multi07:42:462
37594052194,9sleep20-21swapper/207:05:322
376418599189,7cyclictest3971631-21tr10:37:342
3764185991814,3cyclictest3910683-21aten2_r5power_e09:37:272
3764185991814,3cyclictest0-21swapper/209:57:442
3764185991814,2cyclictest0-21swapper/209:17:362
3764185991813,3cyclictest3959909-21unixbench_singl10:22:472
3764185991813,3cyclictest0-21swapper/209:55:012
3764185991813,3cyclictest0-21swapper/208:22:322
3764185991810,6cyclictest3963790-21tr10:27:352
3764185991713,2cyclictest0-21swapper/211:35:092
3764185991713,2cyclictest0-21swapper/210:52:312
3764185991713,2cyclictest0-21swapper/209:47:372
3764185991713,2cyclictest0-21swapper/208:45:542
3764185991713,2cyclictest0-21swapper/208:00:002
3764185991712,4cyclictest3904178-21needreboot09:27:372
3764185991711,4cyclictest0-21swapper/208:27:342
376418599171,12cyclictest3979897-21apt-get10:44:392
376418599170,15cyclictest3955244-21true10:20:012
376418599170,15cyclictest3846585-21irqcore08:32:342
376418599168,5cyclictest3897528-21kthreadcore09:22:342
376418599167,7cyclictest4085561-21grep12:32:312
376418599167,7cyclictest4085561-21grep12:32:302
376418599162,5cyclictest0-21swapper/210:12:312
376418599162,10cyclictest3866128-21diskstats08:52:302
3764185991613,2cyclictest0-21swapper/211:17:122
3764185991613,2cyclictest0-21swapper/209:00:472
3764185991612,2cyclictest0-21swapper/212:11:112
40803192152,8sleep30-21swapper/312:27:263
40011002156,5sleep30-21swapper/311:07:343
39125762152,9sleep30-21swapper/309:37:353
3764185991511,2cyclictest0-21swapper/211:27:432
38653982146,5sleep30-21swapper/308:49:373
37644792142,8sleep30-21swapper/307:12:243
40897832132,6sleep30-21swapper/312:35:003
40755822133,6sleep30-21swapper/312:22:333
39849532132,6sleep30-21swapper/310:50:013
39643322134,5sleep30-21swapper/310:27:383
38699052132,7sleep30-21swapper/308:52:463
38108332132,6sleep30-21swapper/307:55:003
38002442132,7sleep30-21swapper/307:42:413
37902802132,7sleep30-21swapper/307:32:433
37860872132,6sleep30-21swapper/307:30:013
40551952122,6sleep30-21swapper/312:02:323
40523542122,6sleep30-21swapper/311:57:373
40403362123,6sleep30-21swapper/311:47:313
40355312122,6sleep30-21swapper/311:42:333
40301582122,6sleep30-21swapper/311:37:273
40206832122,6sleep30-21swapper/311:27:323
40154722122,6sleep30-21swapper/311:22:313
40100162122,6sleep30-21swapper/311:16:173
39990022122,6sleep30-21swapper/311:02:413
39841922122,6sleep30-21swapper/310:47:403
39841922122,6sleep30-21swapper/310:47:403
39709662122,6sleep30-21swapper/310:37:333
39655372122,6sleep30-21swapper/310:32:283
39593872122,6sleep30-21swapper/310:22:413
39413002122,6sleep30-21swapper/310:07:323
39365032122,6sleep30-21swapper/310:02:353
39299822122,7sleep30-21swapper/309:55:003
39050112122,6sleep30-21swapper/309:30:013
38908642122,6sleep30-21swapper/309:17:293
38845532122,6sleep30-21swapper/309:07:423
38595782122,6sleep30-21swapper/308:42:393
38548502122,6sleep30-21swapper/308:37:453
38468052122,6sleep30-21swapper/308:32:363
38403212122,6sleep30-21swapper/308:22:493
38162632122,6sleep30-21swapper/308:02:283
37914822123,6sleep30-21swapper/307:37:273
37757812122,6sleep30-21swapper/307:17:453
40852762112,5sleep30-21swapper/312:32:283
40852762112,5sleep30-21swapper/312:32:283
40697812112,6sleep30-21swapper/312:15:153
40645942112,5sleep30-21swapper/312:10:013
40600952112,5sleep30-21swapper/312:07:283
40448582113,5sleep30-21swapper/311:50:553
40247982112,5sleep30-21swapper/311:30:013
39901562112,5sleep30-21swapper/310:55:393
39754262112,5sleep30-21swapper/310:42:273
39504792112,6sleep30-21swapper/310:15:113
39455712112,6sleep30-21swapper/310:10:333
39343842112,5sleep30-21swapper/309:57:373
39208252112,5sleep30-21swapper/309:47:303
39188562112,6sleep30-21swapper/309:42:373
39036602112,6sleep30-21swapper/309:27:363
38994412112,6sleep30-21swapper/309:22:433
38859912112,5sleep30-21swapper/309:12:323
38757252112,6sleep30-21swapper/309:02:133
38707502112,5sleep30-21swapper/308:56:393
38348202112,5sleep30-21swapper/308:17:393
38306092113,5sleep30-21swapper/308:14:543
38211452112,5sleep30-21swapper/308:07:243
38016752112,5sleep30-21swapper/307:47:313
37777052112,6sleep30-21swapper/307:22:353
38409532102,5sleep30-21swapper/308:27:263
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional