You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-27 - 04:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Fri Mar 27, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19844862780,5chrt0-21swapper/320:46:363
21802702660,5sleep0141ktimers/000:30:010
19805722650,5sleep30-21swapper/320:41:383
19184562560,3sleep00-21swapper/019:41:360
1885053995243,6cyclictest280-21systemd-journal21:31:301
1885057995043,5cyclictest280-21systemd-journal23:00:002
1885053995041,7cyclictest1981786-21kworker/u8:1+flush-8:023:31:381
1885051995046,2cyclictest1870224-21kworker/u8:1+events_unbound19:15:040
1885053994534,8cyclictest2154296-21ntpq00:01:421
1885053994437,5cyclictest1996408-21latency_hist21:00:001
1885053994437,5cyclictest1996408-21latency_hist21:00:001
1885053994437,5cyclictest1875354-21kworker/u8:0+events_unbound19:16:451
21622882430,2sleep10-21swapper/100:11:361
1885057994337,4cyclictest2149903-21awk00:00:012
1885057994337,4cyclictest2149903-21awk00:00:012
1885053994339,3cyclictest1039035-21kworker/u8:3+flush-8:020:01:301
1885053994337,5cyclictest2094958-21kworker/u8:3+events_unbound23:11:451
1885051994132,7cyclictest2064377-21ntpq22:31:400
21088522400,2sleep3481ktimers/323:16:363
1885057994032,5cyclictest2060042-21turbostat.cron22:30:002
1885051994035,4cyclictest1039035-21kworker/u8:3+flush-8:020:31:310
1885053993928,8cyclictest2189206-21ntpq00:36:421
1885053993833,4cyclictest1039035-21kworker/u8:3+flush-8:020:11:341
1885053993832,4cyclictest2013774-21kthreadcore21:16:351
1885053993832,4cyclictest1875354-21kworker/u8:0+events_unbound19:41:421
1885053993830,6cyclictest2155499-21apt-get00:06:261
1885051993829,7cyclictest2099891-21latency_hist23:10:000
19159852370,8sleep1281rcuc/119:36:391
1885053993732,3cyclictest2175845-21cut00:26:311
1885053993732,3cyclictest2054125-21kworker/u8:3+events_unbound22:45:001
1885053993731,4cyclictest1039035-21kworker/u8:3+events_unbound20:21:281
1885051993733,3cyclictest1039035-21kworker/u8:3+events_unbound19:31:330
1885057993630,4cyclictest1981786-21kworker/u8:1+events_unbound22:51:312
1885053993631,3cyclictest2109960-21kworker/u8:0+flush-8:023:46:391
1885053993631,3cyclictest1911736-21kworker/u8:1+events_unbound19:46:301
1885053993629,5cyclictest280-21systemd-journal23:42:301
1885053993629,5cyclictest280-21systemd-journal22:25:301
1885053993625,9cyclictest2053632-21apt-get22:15:001
1885051993631,3cyclictest2046781-21kworker/u8:2+flush-8:021:55:310
1885051993628,6cyclictest1981786-21kworker/u8:1+flush-8:021:31:380
18847542364,9sleep10-21swapper/119:08:311
1885053993530,4cyclictest1039035-21kworker/u8:3+flush-8:019:26:371
1885057993427,5cyclictest2114897-21journal-offline23:24:222
1885057993424,8cyclictest2053931-21cat22:15:012
1885053993431,2cyclictest1952961-21irqrtprio20:16:331
1885053993430,3cyclictest1981786-21kworker/u8:1+flush-8:021:10:311
1885053993429,4cyclictest2130278-21kworker/u8:4+flush-8:023:51:391
1885053993428,4cyclictest1991466-21kworker/u8:0+events_unbound21:01:451
1885051993430,3cyclictest2150402-21kworker/u8:3+flush-8:000:11:370
1885053993329,3cyclictest1039035-21kworker/u8:3+flush-8:020:31:451
1885053993328,3cyclictest2074882-21kworker/u8:2+flush-8:022:46:341
1885053993327,4cyclictest280-21systemd-journal19:32:261
1885053993324,7cyclictest2170599-21/usr/sbin/munin00:21:421
1885051993327,4cyclictest1039035-21kworker/u8:3+events_unbound20:01:450
1885051993325,6cyclictest1947841-21irqcore20:11:360
1885051993325,6cyclictest1922133-21/usr/sbin/munin19:46:380
1885053993228,2cyclictest0-21swapper/123:10:001
1885053993227,4cyclictest2054125-21kworker/u8:3+flush-8:022:31:321
1885053993227,3cyclictest2047804-21cron22:05:001
1885053993227,3cyclictest0-21swapper/120:40:001
1885053993225,5cyclictest1981786-21kworker/u8:1+flush-8:022:51:471
1885053993225,5cyclictest1901751-21cron19:25:001
1885053993223,7cyclictest1991436-21cron20:55:001
1885053993217,2cyclictest0-21swapper/119:51:281
1885051993228,2cyclictest2071136-21irqcore22:41:340
1885051993226,4cyclictest1911736-21kworker/u8:1+flush-8:019:36:440
1885051993220,9cyclictest280-21systemd-journal00:36:410
1885051993218,12cyclictest2105010-21cron23:15:000
1885053993126,4cyclictest1956838-21kworker/u8:2+events_unbound20:41:401
1885053993126,3cyclictest2119906-21cron23:30:001
1885051993127,2cyclictest1985648-21needreboot20:46:380
1885051993124,5cyclictest1906677-21latency_hist19:30:010
1885051993123,6cyclictest1957163-21taskset20:21:280
1885053993025,4cyclictest1962378-21cpuspeed_turbos20:26:301
1885053993025,3cyclictest2016943-21fschecks_time21:21:301
1885053993024,4cyclictest2036530-21aten2_r5power_v21:41:281
1885053993024,4cyclictest1885533-21cpuspeed_turbos19:11:301
1885053993023,5cyclictest1981786-21kworker/u8:1+flush-8:022:06:401
1885051993026,3cyclictest2109960-21kworker/u8:0+events_unbound23:21:330
1885051993024,4cyclictest2119558-21unixbench_multi23:26:470
1885051993024,4cyclictest2021753-21cpuspeed_turbos21:26:290
1885051993023,5cyclictest1911736-21kworker/u8:1+flush-8:020:06:310
20735232290,4sleep3481ktimers/322:41:373
1885053992924,3cyclictest2040992-21kworker/u8:0+flush-8:022:36:441
1885053992924,3cyclictest1946663-21unixbench_singl20:06:461
1885053992923,4cyclictest2130278-21kworker/u8:4+events_unbound23:56:421
1885053992923,4cyclictest2130278-21kworker/u8:4+events_unbound23:56:421
1885053992923,4cyclictest2090599-21cpuspeed_turbos23:01:301
1885053992922,5cyclictest1936586-21cron20:00:001
1885051992925,2cyclictest2128899-21needreboot23:36:390
1885051992924,3cyclictest2149981-21apt-config00:00:010
1885051992924,3cyclictest2149981-21apt-config00:00:010
1885051992924,3cyclictest2053673-21systemd-run22:15:010
1885051992923,5cyclictest2094958-21kworker/u8:3+events_unbound23:16:390
1885051992923,4cyclictest2140031-21latency_hist23:50:000
1885051992923,4cyclictest1875354-21kworker/u8:0+events_unbound19:51:430
1885057992823,3cyclictest1927099-21aten2_r5power_c19:51:282
1885057992823,3cyclictest1907553-21fschecks_count19:31:292
1885053992824,3cyclictest2109960-21kworker/u8:0+flush-8:023:36:441
1885053992824,2cyclictest2042051-21irqrtprio21:46:331
1885053992823,3cyclictest1981786-21kworker/u8:1+events_unbound22:56:431
1885051992824,3cyclictest2089977-21cron23:00:000
1885051992822,4cyclictest2155137-21awk00:05:000
1885057992723,2cyclictest2041386-21aten2_r5power_e21:46:272
1885057992721,4cyclictest2150402-21kworker/u8:3+events_unbound00:11:502
1885053992721,4cyclictest2055118-21cron22:25:001
1885051992724,2cyclictest2121197-21kthreadcore23:31:330
1885051992724,2cyclictest2011220-21cron21:15:000
1885051992722,4cyclictest2016881-21fschecks_count21:21:300
1885051992722,4cyclictest1981786-21kworker/u8:1+flush-8:021:16:450
1885051992722,3cyclictest2150402-21kworker/u8:3+flush-8:000:05:300
1885051992722,3cyclictest2090681-21fschecks_time23:01:310
1885051992722,3cyclictest1870224-21kworker/u8:1+events_unbound19:10:010
1885051992721,4cyclictest1991133-21unixbench_singl20:51:470
1885051992717,9cyclictest280-21systemd-journal20:30:000
20696852260,6sleep30-21swapper/322:36:473
1885057992622,2cyclictest2070811-21fschecks_count22:41:312
1885057992621,3cyclictest1981786-21kworker/u8:1+events_unbound21:18:422
1885057992620,4cyclictest1992068-21fschecks_count20:56:292
1885057992620,4cyclictest1992068-21fschecks_count20:56:292
1885057992616,8cyclictest2135053-21cat23:45:002
1885053992622,3cyclictest2109960-21kworker/u8:0+events_unbound23:21:311
1885053992622,2cyclictest0-21swapper/100:16:331
1885053992622,2cyclictest0-21swapper/100:16:331
1885053992621,4cyclictest1956838-21kworker/u8:2+flush-8:020:46:401
1885053992620,4cyclictest2105602-21cpuspeed_turbos23:16:291
1885053992620,4cyclictest2010702-21kworker/u8:3+events_unbound21:26:331
1885051992622,2cyclictest0-21swapper/000:16:320
1885051992622,2cyclictest0-21swapper/000:16:320
1885051992620,4cyclictest20977-21cron00:35:000
1885057992517,6cyclictest280-21systemd-journal20:05:262
1885053992521,2cyclictest0-21swapper/121:36:451
1885051992522,2cyclictest1936604-21latency_hist20:00:010
1885051992520,4cyclictest2026133-21kworker/u8:2+flush-8:021:36:310
1885051992520,3cyclictest2074882-21kworker/u8:2+flush-8:022:46:490
1885051992519,4cyclictest2140674-21fschecks_time23:51:300
1885051992519,4cyclictest2130399-21apt-get23:41:250
1885051992519,4cyclictest1875354-21kworker/u8:0+events_unbound19:21:390
1885051992516,7cyclictest2054477-21cron22:20:000
1885051992516,7cyclictest1976252-21unixbench_multi20:36:480
1885057992420,3cyclictest2006684-21aten2_r5power_c21:11:272
1885057992420,3cyclictest1941996-21latency_hist20:05:012
1885057992420,2cyclictest2170656-21aten2_r5power_c00:21:292
1885057992418,4cyclictest1956838-21kworker/u8:2+flush-8:021:05:302
1885057992415,7cyclictest1936964-21cat20:00:012
1885057992414,8cyclictest2040980-21cat21:45:002
1885053992418,4cyclictest2047163-21cron22:00:001
1885051992420,2cyclictest0-21swapper/022:10:000
1885051992419,4cyclictest1981786-21kworker/u8:1+events_unbound22:51:460
1885051992419,3cyclictest1997073-21fschecks_count21:01:300
1885051992419,3cyclictest0-21swapper/020:56:320
1885051992419,3cyclictest0-21swapper/020:56:320
1885051992418,4cyclictest1981545-21cron20:45:000
1885051992418,4cyclictest1952505-21fschecks_count20:16:300
1885051992417,6cyclictest2040962-21cron21:45:000
20112632230,6sleep30-21swapper/321:15:013
1885057992318,3cyclictest2155556-21aten2_r5power_e00:06:282
1885057992317,4cyclictest2130217-21base6423:40:012
1885057992317,4cyclictest2036037-21sh21:40:002
188505799230,7cyclictest20296311kthreadcore21:31:352
1885053992319,2cyclictest0-21swapper/122:20:001
1885053992317,4cyclictest1991466-21kworker/u8:0+flush-8:021:06:351
1885053992311,2cyclictest21815881kthreadcore00:31:371
1885051992319,2cyclictest2045953-21systemd-run21:50:000
1885051992317,4cyclictest2069820-21unixbench_multi22:36:480
20183852220,2sleep375-21kworker/3:1-mm_percpu_wq21:21:353
1885057992218,3cyclictest2130278-21kworker/u8:4+events_unbound00:26:492
1885057992213,7cyclictest2120032-21sort23:30:002
1885051992219,2cyclictest1885045-21cyclictest22:22:300
1885051992216,4cyclictest2174883-21unixbench_multi00:21:470
1885051992214,7cyclictest1889892-21cron19:15:000
1885057992117,2cyclictest0-21swapper/219:30:012
1885057992116,3cyclictest1885341-21aten2_r5power_c19:11:292
188505199212,12cyclictest583-21snmpd21:06:260
1885051992116,4cyclictest2054125-21kworker/u8:3+events_unbound22:26:380
1885057992017,2cyclictest1947577-21fschecks_count20:11:332
1885057992015,4cyclictest1981786-21kworker/u8:1+events_unbound22:17:292
1885057992014,4cyclictest0-21swapper/200:16:332
1885057992014,4cyclictest0-21swapper/200:16:332
1885057992012,6cyclictest2053160-21cut22:10:012
18846112204,9sleep20-21swapper/219:06:472
21434902190,2sleep3481ktimers/323:51:363
188505799199,7cyclictest1996793-21cut21:01:282
188505799196,9cyclictest1986894-21apt-get20:51:282
188505799192,12cyclictest2021526-21apt-get21:26:282
188505799192,12cyclictest1917203-21apt-get19:41:292
1885057991915,2cyclictest0-21swapper/223:10:002
1885057991914,4cyclictest1870224-21kworker/u8:1+flush-8:019:16:462
1885057991913,4cyclictest2054125-21kworker/u8:3+flush-8:022:45:062
1885057991911,6cyclictest2145005-21cat23:55:012
1885053991915,2cyclictest0-21swapper/121:55:001
1885051991915,2cyclictest0-21swapper/021:55:000
1885051991913,4cyclictest0-21swapper/022:05:010
18842282193,8sleep30-21swapper/319:06:443
21746252180,2sleep3471rcuc/300:21:463
1885057991814,2cyclictest2016648-21aten2_r5power_e21:21:282
1885057991813,3cyclictest0-21swapper/220:30:002
1885057991813,3cyclictest0-21swapper/200:36:382
1885057991812,4cyclictest2181234-21tr00:31:362
188505799179,6cyclictest2150813-21fschecks_count00:01:332
1885057991713,3cyclictest1981786-21kworker/u8:1+flush-8:022:31:422
1885057991713,2cyclictest1977273-21fschecks_count20:41:312
1885057991713,2cyclictest0-21swapper/223:31:292
1885057991713,2cyclictest0-21swapper/219:36:392
1885057991711,5cyclictest1971442-21kworker/u8:0+events_unbound20:45:052
188505799167,7cyclictest2101114-21sed23:11:342
188505799167,7cyclictest2068885-21tr22:36:382
188505799164,9cyclictest1897156-21acpi19:21:252
1885057991613,2cyclictest0-21swapper/223:00:442
1885057991613,2cyclictest0-21swapper/222:01:132
1885057991613,2cyclictest0-21swapper/221:59:152
1885057991613,2cyclictest0-21swapper/219:45:042
1885057991612,2cyclictest0-21swapper/223:16:282
1885057991612,2cyclictest0-21swapper/220:16:332
1885057991611,3cyclictest0-21swapper/223:46:352
1885057991611,3cyclictest0-21swapper/220:21:392
188505799161,11cyclictest1972003-21apt-get20:36:272
20362622155,6sleep30-21swapper/321:40:023
1885057991511,3cyclictest2055132-21munin-plugin-st22:25:002
188505799151,11cyclictest1970955-21ps20:31:452
21251912142,8sleep30-21swapper/323:36:283
188505799148,4cyclictest0-21swapper/221:55:002
21645452132,7sleep30-21swapper/300:11:433
21562132132,7sleep30-21swapper/300:06:383
21197832132,7sleep30-21swapper/323:28:293
20769682133,6sleep30-21swapper/322:46:363
20250522134,6sleep30-21swapper/321:26:383
19969432132,7sleep30-21swapper/321:01:303
21808892122,6sleep30-21swapper/300:31:343
21793072122,6sleep30-21swapper/300:26:403
21694382123,6sleep30-21swapper/300:16:413
21694382123,6sleep30-21swapper/300:16:413
21533072122,6sleep30-21swapper/300:01:403
21449402122,5sleep30-21swapper/323:55:023
21449402122,5sleep30-21swapper/323:55:023
21205942122,6sleep30-21swapper/323:31:303
20952622122,6sleep30-21swapper/323:06:273
20536932122,5sleep30-21swapper/322:15:013
20459492122,5sleep30-21swapper/321:50:013
20418032122,6sleep30-21swapper/321:46:323
20302852122,6sleep30-21swapper/321:31:393
20112632122,6sleep30-21swapper/321:15:023
19867582122,5sleep30-21swapper/320:50:023
19716642122,5sleep30-21swapper/320:35:013
19619962122,6sleep30-21swapper/320:25:023
19480762122,6sleep30-21swapper/320:11:373
19427652122,6sleep30-21swapper/320:06:323
19372082122,6sleep30-21swapper/320:01:263
19316482123,5sleep30-21swapper/319:55:003
19226552122,6sleep30-21swapper/319:46:333
19158372122,6sleep30-21swapper/319:36:383
19077632122,6sleep30-21swapper/319:31:323
19019842122,5sleep30-21swapper/319:25:023
18925502122,6sleep30-21swapper/319:16:323
18874772123,5sleep30-21swapper/319:11:383
21881562112,5sleep30-21swapper/300:36:393
21356552112,5sleep30-21swapper/323:46:313
21345482112,5sleep30-21swapper/323:41:463
21103442114,4sleep30-21swapper/323:21:273
21003452112,5sleep30-21swapper/323:10:033
20941972112,5sleep30-21swapper/323:01:453
20852782112,5sleep30-21swapper/322:55:023
20808702112,5sleep30-21swapper/322:51:333
20594842112,5sleep30-21swapper/322:26:493
20547682112,5sleep30-21swapper/322:20:313
20532302112,5sleep30-21swapper/322:10:043
20480402112,5sleep30-21swapper/322:05:023
20315082112,5sleep30-21swapper/321:36:293
20057632110,6sleep30-21swapper/321:06:473
19922582112,5sleep30-21swapper/320:56:323
19922582112,5sleep30-21swapper/320:56:323
19669712112,6sleep30-21swapper/320:30:323
19574182112,5sleep30-21swapper/320:21:333
19565142112,5sleep30-21swapper/320:16:473
19319712112,6sleep30-21swapper/319:56:083
19171772112,5sleep30-21swapper/319:41:283
18973242112,5sleep30-21swapper/319:21:293
20604962100,5sleep30-21swapper/322:30:023
20474192102,5sleep30-21swapper/322:00:053
20467922102,5sleep30-21swapper/321:55:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional