You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-24 - 05:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Tue Mar 24, 2026 00:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
281975421880,2sleep3471rcuc/323:26:403
257179021020,5chrt0-21swapper/319:16:433
26147432880,2sleep3481ktimers/320:01:383
27452362640,5sleep20-21swapper/222:11:392
27636482610,5chrt0-21swapper/322:31:383
28888302600,8chrt75-21kworker/3:1+mm_percpu_wq00:36:383
27377522590,5sleep20-21swapper/222:06:372
2560604995847,8cyclictest2756040-21latency_hist22:25:000
2560606995443,8cyclictest2632209-21latency_hist20:20:011
2560604995144,5cyclictest2587075-21kworker/u8:2+events_unbound21:46:470
2560606995040,7cyclictest2886308-21latency_hist00:35:011
2560606994831,14cyclictest2770844-21latency_hist22:40:021
2560606994738,7cyclictest2755192-21ntpq22:21:421
2560606994640,5cyclictest2736030-21kworker/u8:0+flush-8:023:56:471
2560606994636,7cyclictest2627019-21latency_hist20:15:011
2560606994536,7cyclictest2821364-21cp23:30:001
2560604994534,8cyclictest280-21systemd-journal00:05:010
2560606994433,8cyclictest2656948-21latency_hist20:45:001
2560604994437,5cyclictest2647052-21kworker/u8:1+flush-8:021:11:350
2560606994329,12cyclictest2810304-21ntpq23:16:411
2560604994131,8cyclictest2855515-21ntpq00:01:410
2560604994032,6cyclictest2791110-21latency_hist23:00:010
2560604994030,8cyclictest2779898-21ntpq22:46:410
26084872390,3sleep20-21swapper/219:56:352
256060699399,19cyclictest0-21swapper/120:33:081
2560606993932,5cyclictest2876821-21apt-get00:26:281
2560606993930,6cyclictest2662114-21latency_hist20:50:001
2560606993930,6cyclictest2662114-21latency_hist20:50:001
2560606993928,8cyclictest280-21systemd-journal22:07:421
2560604993930,7cyclictest2702288-21diskstats21:31:330
2560613993831,5cyclictest2780725-21apt-get22:50:002
2560606993826,10cyclictest2618098-21irqcore20:06:371
28336132370,5sleep3481ktimers/323:41:363
2560606993725,8cyclictest280-21systemd-journal23:47:411
2560604993731,4cyclictest1541308-21kworker/u8:1+flush-8:019:16:430
2560604993725,9cyclictest2795459-21ntpq23:01:380
2560604993725,8cyclictest2627018-21apt-get20:15:000
2560606993630,5cyclictest2647052-21kworker/u8:1+flush-8:021:11:461
2560606993626,8cyclictest2815461-21ntpq23:21:411
2560604993632,3cyclictest1744248-21kworker/u8:4+flush-8:020:06:370
2560604993632,3cyclictest1744248-21kworker/u8:4+flush-8:019:36:320
2560606993530,3cyclictest2706629-21sh21:35:001
256060699343,5cyclictest0-21swapper/123:36:291
2560606993423,9cyclictest2691804-21idleruntime-cro21:20:001
2560606993421,11cyclictest2577046-21unixbench_multi19:21:491
2560606993418,14cyclictest2677006-21latency_hist21:05:011
2560606993417,15cyclictest2791717-21apt-get23:01:251
2560604993430,3cyclictest2786353-21latency_hist22:55:010
2560604993427,5cyclictest2647052-21kworker/u8:1+events_unbound20:55:000
2560604993425,7cyclictest2647643-21diskstats20:36:340
2560604993421,10cyclictest280-21systemd-journal21:36:410
256060499341,30cyclictest2811165-21munin-run23:20:010
256060699339,11cyclictest0-21swapper/121:05:461
256060699334,20cyclictest0-21swapper/121:37:571
256060699333,5cyclictest0-21swapper/119:16:261
2560606993328,4cyclictest2736030-21kworker/u8:0+flush-8:023:31:471
2560606993324,6cyclictest2871455-21latency_hist00:20:001
2560606993324,6cyclictest2871455-21latency_hist00:20:001
2560606993323,8cyclictest0-21swapper/100:05:001
2560606993317,13cyclictest280-21systemd-journal19:51:421
2560606993315,16cyclictest2784675-21sshd22:51:421
2560604993327,4cyclictest2587075-21kworker/u8:2+events_unbound22:26:420
2560604993324,7cyclictest280-21systemd-journal23:25:000
256060699323,5cyclictest0-21swapper/100:36:291
2560606993211,18cyclictest0-21swapper/121:56:451
27076442310,5sleep30-21swapper/321:36:363
256060699313,5cyclictest0-21swapper/123:51:301
256060699313,5cyclictest0-21swapper/119:41:301
256060699313,5cyclictest0-21swapper/119:26:301
2560606993125,4cyclictest2587608-21apt-get19:36:271
2560606993123,5cyclictest2560439-21latency_hist19:10:001
2560606993122,6cyclictest2775800-21latency_hist22:45:011
2560606993122,6cyclictest2765173-21ntpq22:31:451
2560606993121,8cyclictest2611368-21ntpq19:56:401
2560606993118,11cyclictest2636854-21unixbench_singl20:21:501
2560606993117,12cyclictest2711949-21apt-get21:41:271
2560606993117,12cyclictest2586921-21unixbench_multi19:31:481
2560606993117,12cyclictest0-21swapper/121:46:461
2560606993114,15cyclictest0-21swapper/122:16:491
2560613993025,4cyclictest1541308-21kworker/u8:1+flush-8:019:15:052
2560606993026,2cyclictest2875578-21needreboot00:21:421
2560606993017,11cyclictest0-21swapper/120:26:341
2560606993017,10cyclictest0-21swapper/122:11:421
2560606993012,16cyclictest2859146-21kthreadcore00:06:371
2560606993012,16cyclictest2760987-21systemd-run22:30:001
2560606993012,16cyclictest0-21swapper/122:46:291
2560604993027,2cyclictest2656991-21systemd-run20:45:000
2560604993026,3cyclictest2587075-21kworker/u8:2+flush-8:020:26:340
2560604993024,4cyclictest2587075-21kworker/u8:2+flush-8:020:19:370
2560604993024,4cyclictest1744248-21kworker/u8:4+events_unbound19:41:350
2560604993021,7cyclictest2875652-21ntpq00:21:440
2560613992924,3cyclictest2656588-21unixbench_multi20:41:502
256060699299,18cyclictest2651147-21ntpq20:36:431
256060699297,14cyclictest26147462cut20:01:381
256060699291,6cyclictest2785949-21apt-get22:55:031
2560606992914,13cyclictest2866499-21latency_hist00:15:001
2560604992924,4cyclictest2864352-21kworker/u8:2+events_unbound00:26:420
2560604992924,3cyclictest2736030-21kworker/u8:0+events_unbound23:06:320
2560604992922,2cyclictest421rcuog/223:31:420
28241572280,1sleep375-21kworker/3:1-events23:31:373
2560613992824,3cyclictest2760970-21turbostat.cron22:30:002
2560613992822,4cyclictest2876460-21idleruntime-cro00:25:002
256060699289,16cyclictest2692429-21fschecks_count21:21:331
2560606992811,15cyclictest0-21swapper/123:41:351
2560606992811,15cyclictest0-21swapper/123:11:391
2560606992811,15cyclictest0-21swapper/120:56:471
2560606992811,14cyclictest0-21swapper/122:01:311
2560604992823,4cyclictest1541308-21kworker/u8:1+flush-8:019:21:340
2560604992822,4cyclictest2846186-21apt-get23:55:000
256060499281,24cyclictest2722212-21hddtemp_smartct21:51:340
256060699274,5cyclictest0-21swapper/120:51:301
256060699271,8cyclictest583-21snmpd21:26:361
2560606992712,13cyclictest0-21swapper/119:50:001
256060699271,11cyclictest2796678-21apt-get23:06:311
256060499272,9cyclictest27622132sed22:31:360
2560604992722,4cyclictest2616314-21sshd20:01:410
2560604992722,3cyclictest2821273-21latency_hist23:30:010
2560604992721,4cyclictest2647052-21kworker/u8:1+flush-8:020:56:350
2560604992719,6cyclictest2690834-21sshd21:16:420
2560604992718,7cyclictest2581481-21ntpq19:26:420
2560604992715,9cyclictest280-21systemd-journal00:00:000
2560613992622,2cyclictest2812001-21fschecks_count23:21:332
2560613992621,3cyclictest2797028-21fschecks_time23:06:332
2560613992621,3cyclictest2662557-21aten2_r5power_p20:51:332
2560613992620,4cyclictest2631902-21unixbench_singl20:16:502
256060699264,5cyclictest2561029-21ls19:11:321
256060699264,5cyclictest2561029-21ls19:11:321
256060699261,11cyclictest2721839-21apt-get21:51:301
2560604992621,3cyclictest0-21swapper/000:35:010
2560604992620,4cyclictest2775436-21unixbench_multi22:41:480
2560604992620,3cyclictest0-21swapper/023:11:430
2560604992619,5cyclictest2602498-21chrt19:51:320
2560613992522,2cyclictest2827108-21fschecks_count23:36:332
2560613992522,2cyclictest2761442-21aten2_r5power_v22:31:312
2560613992521,2cyclictest2702304-21fschecks_count21:31:332
2560613992515,8cyclictest2736293-21awk22:05:002
2560604992520,3cyclictest2831975-21cpuspeed_turbos23:41:300
2560604992520,2cyclictest0-21swapper/022:01:420
2560604992519,4cyclictest2560461-21kworker/u8:0+flush-8:019:31:460
26366602240,4sleep0201rcuc/020:21:490
2560613992420,2cyclictest2850999-21unixbench_multi23:56:492
2560613992420,2cyclictest2717149-21fschecks_count21:46:332
2560613992419,4cyclictest2750754-21unixbench_multi22:16:502
2560613992419,3cyclictest2862162-21cpuspeed_turbos00:11:332
2560613992419,3cyclictest2755733-21unixbench_singl22:21:502
2560613992418,4cyclictest2676582-21unixbench_multi21:01:492
256060499242,21cyclictest0-21swapper/022:06:470
2560604992419,4cyclictest2646177-21ntp_kernel_err20:31:420
2560604992419,3cyclictest401-21dbus-daemon22:40:010
2560604992419,3cyclictest2751126-21idleruntime-cro22:20:000
2560604992418,5cyclictest1744248-21kworker/u8:4+events_unbound20:50:000
2560604992418,5cyclictest1744248-21kworker/u8:4+events_unbound20:50:000
2560604992418,4cyclictest1744248-21kworker/u8:4+flush-8:019:46:470
2560604992416,6cyclictest2841258-21latency_hist23:50:010
2560604992416,6cyclictest2830585-21ntpq23:36:440
2560604992415,6cyclictest2612222-21latency_hist20:00:010
25559262242,15sleep02555977-21wget19:06:300
28049902230,2sleep375-21kworker/3:1-events23:11:393
2560613992319,2cyclictest2881060-21unixbench_singl00:26:492
2560613992318,3cyclictest2651641-21unixbench_multi20:36:502
2560613992312,8cyclictest2791107-21awk23:00:002
2560604992318,3cyclictest2696376-21unixbench_multi21:21:500
2560604992317,5cyclictest1744248-21kworker/u8:4+flush-8:021:41:370
2560604992317,4cyclictest2736030-21kworker/u8:0+flush-8:000:15:470
2560604992317,4cyclictest2736030-21kworker/u8:0+flush-8:000:15:470
2560604992316,5cyclictest2890435-21ntpq00:36:420
2560604992311,4cyclictest2587075-21kworker/u8:2+flush-8:021:00:310
2560613992219,2cyclictest2616789-21unixbench_multi20:01:482
2560613992218,2cyclictest2822088-21cpuspeed_turbos23:31:312
2560613992218,2cyclictest0-21swapper/222:55:012
2560613992217,4cyclictest2791574-21cp23:00:012
2560613992215,5cyclictest0-21swapper/223:30:012
2560613992212,7cyclictest2607364-21date19:55:012
256060499222,19cyclictest0-21swapper/021:06:480
2560604992218,2cyclictest0-21swapper/021:56:380
2560604992217,3cyclictest2741624-21aten2_r5power_c22:11:290
2560604992217,3cyclictest0-21swapper/019:11:380
2560604992217,3cyclictest0-21swapper/019:11:380
256060499220,8cyclictest2862315-21/usr/sbin/munin00:11:350
256061399212,13cyclictest2692168-21apt-get21:21:302
2560613992117,2cyclictest0-21swapper/219:31:462
2560613992116,3cyclictest2577375-21sh19:25:002
2560613992114,5cyclictest280-21systemd-journal00:09:422
2560613992114,3cyclictest0-21swapper/219:26:472
2560613992111,8cyclictest2598217-21sed19:46:352
2560613992111,7cyclictest2836534-21sed23:45:012
256060499212,14cyclictest583-21snmpd21:26:310
256061399202,7cyclictest0-21swapper/220:26:192
2560613992017,2cyclictest2852674-21sed00:01:372
2560613992017,2cyclictest2836713-21aten2_r5power_c23:46:312
2560613992017,2cyclictest2681838-21cron21:10:002
2560613992017,2cyclictest2671338-21find20:56:492
2560613992016,2cyclictest0-21swapper/223:20:012
2560613992013,5cyclictest280-21systemd-journal20:34:072
2560613992012,6cyclictest2591456-21sshd19:36:422
2560613992011,7cyclictest2561552-21sed19:11:362
2560613992011,7cyclictest2561552-21sed19:11:362
256061399200,17cyclictest2843117-21tr23:51:372
256061399195,3cyclictest0-21swapper/221:51:332
2560613991915,2cyclictest2657884-21fschecks_time20:46:312
2560613991915,2cyclictest2657884-21fschecks_time20:46:312
2560613991915,2cyclictest0-21swapper/222:36:372
2560613991915,2cyclictest0-21swapper/221:20:002
2560613991914,3cyclictest0-21swapper/221:11:412
2560613991914,3cyclictest0-21swapper/220:11:482
2560613991911,6cyclictest2707039-21cut21:36:312
2560613991910,7cyclictest2715108-21tr21:41:382
25561592194,9sleep20-21swapper/219:06:322
256061399182,6cyclictest0-21swapper/223:10:192
2560613991814,3cyclictest0-21swapper/222:45:002
2560613991814,2cyclictest0-21swapper/219:41:222
2560613991812,4cyclictest2699865-21cut21:26:392
2560613991811,5cyclictest2727555-21sed21:56:362
25603572183,9sleep30-21swapper/319:09:103
2560613991714,2cyclictest2867121-21fschecks_count00:16:342
2560613991714,2cyclictest2867121-21fschecks_count00:16:342
2560613991713,2cyclictest2617773-21fschecks_count20:06:342
2560613991712,3cyclictest0-21swapper/200:36:372
2560613991710,5cyclictest2883913-21tr00:31:402
2560613991612,2cyclictest0-21swapper/220:21:472
25927042150,3sleep3481ktimers/319:41:313
28808062143,7sleep30-21swapper/300:26:483
27700242132,7sleep30-21swapper/322:36:413
27467782132,7sleep30-21swapper/322:16:343
26967442132,6sleep30-21swapper/321:25:023
26279872132,7sleep30-21swapper/320:16:363
26221052132,6sleep30-21swapper/320:10:013
28724242122,6sleep30-21swapper/300:21:383
28573352122,6sleep30-21swapper/300:06:333
28558002123,6sleep30-21swapper/300:01:483
28466392122,5sleep30-21swapper/323:55:023
28456582122,6sleep30-21swapper/323:51:493
28412422122,6sleep30-21swapper/323:50:003
28292572122,6sleep30-21swapper/323:36:403
28065312122,6sleep30-21swapper/323:16:293
27963432122,5sleep30-21swapper/323:05:023
27833132122,6sleep30-21swapper/322:51:413
27708842122,5sleep30-21swapper/322:40:023
27320482122,6sleep30-21swapper/322:01:333
27215612122,5sleep30-21swapper/321:50:023
27204972122,6sleep30-21swapper/321:46:413
27159572123,5sleep30-21swapper/321:41:473
26867452123,5sleep30-21swapper/321:14:103
26772962122,6sleep30-21swapper/321:06:323
26720122122,6sleep30-21swapper/321:00:023
26677802122,6sleep30-21swapper/320:56:363
26575782123,5sleep30-21swapper/320:46:293
26575782123,5sleep30-21swapper/320:46:293
26526322122,6sleep30-21swapper/320:41:363
26514752122,6sleep30-21swapper/320:36:503
26420812122,6sleep30-21swapper/320:30:003
26079872122,6sleep30-21swapper/319:56:333
26028492122,6sleep30-21swapper/319:51:363
25998442122,6sleep30-21swapper/319:46:383
25874622122,6sleep30-21swapper/319:35:023
25654012122,6sleep30-21swapper/319:14:033
25654012122,6sleep30-21swapper/319:14:033
28860492112,6sleep30-21swapper/300:31:543
28693282112,5sleep30-21swapper/300:16:413
28693282112,5sleep30-21swapper/300:16:413
28661132112,5sleep30-21swapper/300:11:513
28121332112,5sleep30-21swapper/323:21:353
27957372112,5sleep30-21swapper/323:01:453
27869412112,5sleep30-21swapper/322:56:353
27766592112,6sleep30-21swapper/322:46:363
27570822112,5sleep30-21swapper/322:26:373
27513842112,6sleep30-21swapper/322:20:313
27461242112,5sleep30-21swapper/322:14:373
27370212112,5sleep30-21swapper/322:06:353
27272332112,6sleep30-21swapper/321:56:363
27020712112,5sleep30-21swapper/321:31:323
26920572112,6sleep30-21swapper/321:20:243
26874392112,5sleep30-21swapper/321:16:353
26664832112,5sleep30-21swapper/320:51:513
26423902112,6sleep30-21swapper/320:30:563
26328312112,5sleep30-21swapper/320:21:333
26175242112,6sleep30-21swapper/320:06:323
25837482112,5sleep30-21swapper/319:31:373
25777092112,5sleep30-21swapper/319:26:273
25730282112,6sleep30-21swapper/319:21:343
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional