You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-11 - 12:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Thu Dec 11, 2025 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2244908996352,9cyclictest2698871-21systemd-journal19:55:003
2244908996352,9cyclictest2698871-21systemd-journal19:55:003
2244898993934,2cyclictest1-21systemd20:20:010
2244900993830,5cyclictest2468182-21latency_hist22:45:001
2244904993733,3cyclictest2255210-21kworker/u8:0+events_unbound19:55:212
2244904993632,3cyclictest2322601-21kworker/u8:1+events_unbound21:10:152
2244904993630,4cyclictest2698871-21systemd-journal21:05:232
2244904993430,2cyclictest2530478-21kworker/u8:1+flush-259:023:50:002
2244904993426,3cyclictest2452740-21kworker/u8:1+events_unbound23:05:182
2244904993330,2cyclictest2400323-21kworker/u8:2+flush-259:021:55:002
2244904993330,2cyclictest2317117-21kworker/u8:3+flush-259:022:05:002
2244904993330,2cyclictest2317117-21kworker/u8:3+events_unbound20:20:132
2244898993326,5cyclictest2535560-21latency_hist23:50:000
2244908993226,4cyclictest2698871-21systemd-journal19:40:003
2244908993226,4cyclictest2572479-21latency_hist00:25:003
2244904993228,3cyclictest2332858-21kworker/u8:2+flush-259:020:55:002
2244898993228,3cyclictest2332858-21kworker/u8:2+flush-259:020:45:000
2244904993128,2cyclictest2494215-21kworker/u8:3+flush-259:023:10:192
2244904993128,2cyclictest2494215-21kworker/u8:3+events_unbound23:20:172
2244900993125,4cyclictest2546276-21latency_hist00:00:011
2244900993124,5cyclictest2519930-21cat23:35:011
2244904993027,2cyclictest2214158-21kworker/u8:2+flush-259:019:25:202
2244904992926,2cyclictest2296824-21kworker/u8:2+events_unbound20:10:012
2244904992925,3cyclictest2416637-21kworker/u8:1+flush-259:022:15:152
2244900992923,4cyclictest2400546-21/usr/sbin/munin21:40:161
2244904992825,2cyclictest2264888-21kworker/u8:3+events_unbound19:45:002
2244904992824,2cyclictest2400323-21kworker/u8:2+flush-259:021:40:232
2244900992823,3cyclictest2343445-21apt-get20:45:131
2244900992821,5cyclictest2389626-21latency_hist21:30:011
2244898992824,3cyclictest2427211-21apt-get22:05:130
2244908992714,5cyclictest2426951-21latency_hist22:05:003
2244904992723,3cyclictest2494214-21kworker/u8:0+events_unbound23:35:212
2244904992722,3cyclictest2473576-21kworker/u8:3+events_unbound22:50:232
2244900992724,2cyclictest2317117-21kworker/u8:3+flush-259:021:00:001
2244898992724,2cyclictest2394885-21kworker/u8:1+events_unbound21:35:260
2244908992620,4cyclictest2411073-21apt-get21:50:003
2244908992619,5cyclictest2509436-21latency_hist23:25:023
2244904992623,2cyclictest2494215-21kworker/u8:3+events_unbound23:25:252
2244898992622,3cyclictest2458086-21kworker/u8:2+events_unbound23:10:020
2244898992622,3cyclictest2383982-21kworker/u8:2+flush-259:021:24:590
2244898992619,5cyclictest2379343-21latency_hist21:20:000
2244908992520,3cyclictest2573161-21apt-get00:25:133
2244904992522,2cyclictest2426964-21kworker/u8:0+events_unbound22:35:142
2244904992522,2cyclictest2394885-21kworker/u8:1+events_unbound21:35:192
2244904992521,3cyclictest2214158-21kworker/u8:2+flush-259:019:10:252
2244908992419,4cyclictest2494214-21kworker/u8:0+flush-259:023:44:593
2244908992417,5cyclictest2316519-21ntpq20:15:213
2244904992421,2cyclictest2583429-21kworker/u8:4+events_unbound00:38:492
2244904992421,2cyclictest2509658-21kworker/u8:4+flush-259:023:30:242
2244904992421,2cyclictest2255210-21kworker/u8:0+flush-259:019:50:202
2244904992421,2cyclictest2255210-21kworker/u8:0+flush-259:019:50:202
2244904992419,4cyclictest2426964-21kworker/u8:0+events_unbound22:45:252
2244900992419,4cyclictest2358955-21/usr/sbin/munin21:00:231
2244900992419,3cyclictest2494214-21kworker/u8:0+flush-259:023:35:161
2244898992421,2cyclictest2264888-21kworker/u8:3+events_unbound19:40:180
2244898992420,3cyclictest2432321-21kworker/u8:4+events_unbound22:10:230
2244898992420,3cyclictest2383982-21kworker/u8:2+events_unbound21:25:160
2244898992417,5cyclictest2244838-21latency_hist19:10:000
2244908992317,4cyclictest2357842-21ntpq20:55:243
2244904992320,2cyclictest2567353-21kworker/u8:2+events_unbound00:25:162
2244904992320,2cyclictest2494214-21kworker/u8:0+events_unbound00:10:132
2244904992320,2cyclictest2478739-21kworker/u8:4+events_unbound22:55:252
2244904992320,2cyclictest2255210-21kworker/u8:0+flush-259:019:35:182
2244904992319,3cyclictest2494214-21kworker/u8:0+events_unbound23:15:152
2244904992319,3cyclictest2432321-21kworker/u8:4+events_unbound22:20:132
2244898992320,2cyclictest2416637-21kworker/u8:1+events_unbound21:55:010
2244898992320,2cyclictest2317117-21kworker/u8:3+flush-259:020:45:210
2244904992220,2cyclictest2322601-21kworker/u8:1+flush-259:020:30:212
2244904992219,2cyclictest2416637-21kworker/u8:1+events_unbound22:10:252
2244904992217,4cyclictest2463259-21apt-get22:40:102
2244898992219,2cyclictest2264888-21kworker/u8:3+events_unbound20:00:250
2244898992219,2cyclictest2264888-21kworker/u8:3+events_unbound19:25:250
2244898992218,3cyclictest2214158-21kworker/u8:2+events_unbound19:30:160
2244904992118,2cyclictest2317117-21kworker/u8:3+events_unbound21:30:262
2244900992117,3cyclictest2572469-21idleruntime-cro00:25:001
2244900992117,2cyclictest2255210-21kworker/u8:0+flush-259:019:35:001
2244900992116,3cyclictest2250431-21diskstats19:15:131
2244898992118,2cyclictest2494215-21kworker/u8:3+flush-259:000:15:200
2244898992118,2cyclictest2494214-21kworker/u8:0+events_unbound00:10:150
2244898992118,2cyclictest2249703-21kworker/u8:1+flush-259:019:15:210
2244898992117,3cyclictest2578299-21apt-get00:30:130
2244898992117,3cyclictest2494215-21kworker/u8:3+flush-259:023:55:260
2244898992117,3cyclictest2411545-21cp21:50:000
2244908992013,6cyclictest2452740-21kworker/u8:1+flush-259:022:45:003
2244908992013,5cyclictest2441556-21ntpq22:15:203
2244904992017,2cyclictest2452740-21kworker/u8:1+events_unbound23:00:182
2244904992017,2cyclictest2317117-21kworker/u8:3+flush-259:021:55:162
2244904992016,3cyclictest3000061-21in:imuxsock20:45:002
2244904992016,3cyclictest2296824-21kworker/u8:2+events_unbound20:00:222
2244904992015,3cyclictest2541235-21base6423:55:012
2244900992017,2cyclictest2369030-21turbostat.cron21:10:001
2244900992016,3cyclictest2462993-21sh22:40:011
2244900992014,4cyclictest2698871-21systemd-journal20:26:211
2244898992016,3cyclictest2583429-21kworker/u8:4+flush-259:000:35:200
2244898992016,3cyclictest2432321-21kworker/u8:4+events_unbound22:20:200
2244904991917,1cyclictest2432321-21kworker/u8:4+flush-259:022:25:202
2244904991915,3cyclictest2317117-21kworker/u8:3+flush-259:020:45:252
2244904991915,3cyclictest2286847-21kworker/u8:1+events_unbound20:15:132
2244900991915,3cyclictest2348586-21apt-get20:50:131
2244900991915,2cyclictest0-21swapper/121:55:001
2244898991915,3cyclictest2400323-21kworker/u8:2+events_unbound21:40:190
2244898991914,4cyclictest2698871-21systemd-journal20:00:000
2244904991815,2cyclictest2494215-21kworker/u8:3+events_unbound00:19:252
2244904991815,2cyclictest2494214-21kworker/u8:0+events_unbound00:05:182
2244904991814,3cyclictest2400323-21kworker/u8:2+events_unbound22:05:242
2244900991814,3cyclictest2473576-21kworker/u8:3+flush-259:022:50:251
2244900991814,3cyclictest2307135-21kworker/u8:0+flush-259:020:40:001
2244900991814,3cyclictest2302014-21sh20:05:001
2244900991814,3cyclictest2260369-21gpgv19:25:001
2244900991813,3cyclictest2317117-21kworker/u8:3+flush-259:021:55:171
2244898991816,1cyclictest0-21swapper/020:25:000
2244898991814,3cyclictest2458120-21apt-get22:35:110
2244898991814,3cyclictest2452740-21kworker/u8:1+events_unbound22:50:140
2244908991714,2cyclictest0-21swapper/319:25:003
2244908991713,3cyclictest2458086-21kworker/u8:2+flush-259:022:45:173
2244904991714,2cyclictest2264888-21kworker/u8:3+events_unbound19:45:202
2244904991714,2cyclictest2264888-21kworker/u8:3+events_unbound19:45:202
2244904991710,5cyclictest2260344-21latency_hist19:25:012
2244900991712,4cyclictest2698871-21systemd-journal23:11:231
2244900991711,4cyclictest2432090-21latency_hist22:10:011
2244898991714,2cyclictest2478739-21kworker/u8:4+events_unbound23:00:160
2244898991713,3cyclictest2264888-21kworker/u8:3+events_unbound19:45:220
2244898991713,3cyclictest2264888-21kworker/u8:3+events_unbound19:45:220
224490899166,9cyclictest2416637-21kworker/u8:1+flush-259:022:15:013
2244904991613,2cyclictest2317117-21kworker/u8:3+flush-259:020:25:272
2244904991612,3cyclictest2383982-21kworker/u8:2+events_unbound21:25:212
2244900991612,3cyclictest2426964-21kworker/u8:0+flush-259:022:45:261
2244900991611,3cyclictest2698871-21systemd-journal23:28:241
224489899169,5cyclictest2698871-21systemd-journal19:15:000
2244898991613,2cyclictest61450irq/134-enp0s29f1:tx-000:25:160
2244898991613,2cyclictest60650irq/133-enp0s29f1:rx-023:40:290
2244898991613,2cyclictest60650irq/133-enp0s29f1:rx-023:17:140
2244898991613,2cyclictest2214158-21kworker/u8:2+events_unbound19:35:140
2244898991612,3cyclictest2458086-21kworker/u8:2+flush-259:022:45:160
2244898991612,3cyclictest2322601-21kworker/u8:1+events_unbound20:25:210
2244908991512,2cyclictest2541808-21irqcore23:55:153
2244908991512,2cyclictest2432094-21latency_hist22:10:003
2244908991511,3cyclictest2557246-21/usr/sbin/munin00:10:153
2244908991511,3cyclictest0-21swapper/300:35:003
2244904991512,3cyclictest2494214-21kworker/u8:0+events_unbound00:02:232
2244904991511,3cyclictest2317117-21kworker/u8:3+events_unbound21:17:012
224490099159,4cyclictest2499386-21/usr/sbin/munin23:15:161
2244900991512,2cyclictest2530439-21cron23:45:001
2244900991512,2cyclictest2282098-21irqcore19:45:211
2244900991512,2cyclictest2282098-21irqcore19:45:201
2244898991514,1cyclictest60650irq/133-enp0s29f1:rx-023:30:580
2244898991513,1cyclictest60650irq/133-enp0s29f1:rx-022:55:250
2244898991513,1cyclictest60650irq/133-enp0s29f1:rx-022:33:010
2244898991513,1cyclictest60650irq/133-enp0s29f1:rx-021:00:160
2244898991512,2cyclictest2400323-21kworker/u8:2+flush-259:022:00:200
2244898991511,3cyclictest2535837-21apt-get23:50:140
2244898991511,3cyclictest2494214-21kworker/u8:0+events_unbound23:23:240
2244898991511,3cyclictest2264888-21kworker/u8:3+events_unbound19:54:230
2244898991511,3cyclictest2264888-21kworker/u8:3+events_unbound19:54:230
224490899149,3cyclictest2698871-21systemd-journal20:24:563
224490899148,4cyclictest2494214-21kworker/u8:0+flush-259:023:30:003
224490899148,4cyclictest2451804-21ntpq22:25:223
224490899147,5cyclictest2388785-21ntpq21:25:223
2244908991412,1cyclictest0-21swapper/321:45:013
2244908991410,3cyclictest2322601-21kworker/u8:1+flush-259:020:40:163
224490499149,3cyclictest2426964-21kworker/u8:0+events_unbound22:32:302
2244904991411,2cyclictest2322601-21kworker/u8:1+events_unbound21:00:232
2244904991411,2cyclictest2322601-21kworker/u8:1+events_unbound20:35:422
2244904991410,3cyclictest2567353-21kworker/u8:2+events_unbound00:33:512
2244904991410,3cyclictest2214158-21kworker/u8:2+events_unbound19:30:132
224490099149,3cyclictest2317117-21kworker/u8:3+events_unbound20:40:201
224490099146,6cyclictest2437253-21latency_hist22:15:001
2244900991411,2cyclictest2442759-21cpuspeed_turbos22:20:131
2244900991411,2cyclictest0-21swapper/100:04:591
2244900991410,3cyclictest2567353-21kworker/u8:2+events_unbound00:30:171
2244900991410,3cyclictest2322601-21kworker/u8:1+flush-259:021:15:161
2244898991412,1cyclictest60650irq/133-enp0s29f1:rx-020:35:150
2244898991411,2cyclictest60650irq/133-enp0s29f1:rx-021:55:140
2244898991411,2cyclictest60650irq/133-enp0s29f1:rx-020:50:250
2244898991410,3cyclictest60650irq/133-enp0s29f1:rx-022:40:200
224490899139,3cyclictest2474288-21irqcore22:50:183
224490899139,3cyclictest2442429-21kworker/u8:2+flush-259:022:25:003
2244908991310,2cyclictest491ktimers/323:35:013
2244908991310,2cyclictest2584496-21kthreadcore00:35:193
224490499139,3cyclictest2249703-21kworker/u8:1+flush-259:019:19:422
224490099139,2cyclictest2411198-21gpgv21:50:001
2244900991310,2cyclictest2494215-21kworker/u8:3-flush-259:000:40:001
2244898991311,1cyclictest60650irq/133-enp0s29f1:rx-023:27:270
2244898991311,1cyclictest60650irq/133-enp0s29f1:rx-021:08:480
2244898991311,1cyclictest60650irq/133-enp0s29f1:rx-020:30:240
2244898991310,2cyclictest60650irq/133-enp0s29f1:rx-020:10:170
2244898991310,2cyclictest2255210-21kworker/u8:0+events_unbound19:20:140
224490899129,2cyclictest2524887-21unixbench_singl23:35:253
224490899129,2cyclictest2484023-21cpuspeed_turbos23:00:123
224490899129,2cyclictest2368546-21unixbench_multi21:05:243
224490899129,2cyclictest2261096-21cpuspeed_turbos19:25:143
224490899128,3cyclictest2399407-21ntp_kernel_pll_21:35:223
2244908991210,1cyclictest2266578-21irqcore19:30:173
2244908991210,1cyclictest0-21swapper/300:05:003
224490899120,11cyclictest2312310-21awk20:15:003
224490499129,2cyclictest2322601-21kworker/u8:1+events_unbound20:55:442
224490099129,2cyclictest2293219-21kthreadcore19:55:181
224490099127,3cyclictest2317940-21cut20:20:131
224490099126,4cyclictest2698871-21systemd-journal22:35:011
224490099124,7cyclictest2271516-21uniq19:35:131
224490899119,1cyclictest0-21swapper/323:08:243
224490899119,1cyclictest0-21swapper/320:35:023
224490899118,2cyclictest2373701-21unixbench_multi21:10:243
224490899118,2cyclictest2328049-21cpuspeed_turbos20:30:143
224490899118,2cyclictest0-21swapper/323:15:253
224490899117,3cyclictest2347906-21unixbench_singl20:45:253
224490899117,3cyclictest2255210-21kworker/u8:0+flush-259:019:55:173
224490899115,4cyclictest2411073-21apt-get21:50:003
224490899115,4cyclictest1-21systemd21:24:593
224490499117,2cyclictest2379709-21tr21:20:142
224490099118,2cyclictest2307443-21cpuspeed_turbos20:10:131
224490099116,3cyclictest2448262-21grep22:25:171
224490099111,8cyclictest0-21swapper/123:05:231
224490899108,1cyclictest0-21swapper/323:55:003
224490899108,1cyclictest0-21swapper/322:40:013
224490899108,1cyclictest0-21swapper/320:30:003
224490899107,2cyclictest0-21swapper/321:35:003
224490899106,3cyclictest2557030-21kworker/u8:1+flush-259:000:09:593
224490899104,4cyclictest2698871-21systemd-journal19:20:003
224490899104,4cyclictest2562720-21fw_forwarded_lo00:15:163
224490899101,8cyclictest0-21swapper/319:15:003
22449089997,1cyclictest2453062-21cpuspeed_turbos22:30:123
22449089997,1cyclictest0-21swapper/321:00:153
22449089996,2cyclictest2530465-21turbostat23:49:553
22449089995,3cyclictest2297703-21sed20:00:193
22449089995,3cyclictest2280308-21sed19:40:203
22449089994,4cyclictest2495874-21cut23:10:173
22449089985,2cyclictest0-21swapper/319:45:213
22449089985,2cyclictest0-21swapper/319:45:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional