You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-16 - 03:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sat May 16, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2275545995249,2cyclictest2414986-21kworker/u8:1+flush-259:021:25:211
2275550995047,2cyclictest2518770-21kworker/u8:5+events_unbound23:50:243
2275550994744,2cyclictest2311521-21kworker/u8:2+flush-259:020:10:213
2275550994742,3cyclictest2337406-21latency_hist20:10:013
2275542994639,5cyclictest2539091-21latency_hist23:25:010
2275550994340,2cyclictest2414986-21kworker/u8:1+events_unbound23:10:213
2275545994340,2cyclictest2301559-21kworker/u8:1+flush-259:020:30:441
2275542994339,3cyclictest2523926-21apt-get23:10:100
2275545994239,2cyclictest2311521-21kworker/u8:2+flush-259:019:55:301
2275545994239,2cyclictest2275749-21kworker/u8:3+flush-259:019:20:241
2275550994139,1cyclictest2518770-21kworker/u8:5+flush-259:023:30:213
2275550994138,2cyclictest2456879-21kworker/u8:0+flush-259:022:15:003
2275550994037,2cyclictest2425360-21kworker/u8:3+flush-259:021:45:233
2275550994036,3cyclictest2575552-21kworker/u8:1+events_unbound00:20:453
2275550994036,3cyclictest2575552-21kworker/u8:1+events_unbound00:20:443
2275545994037,2cyclictest2285470-21kworker/u8:0+events_unbound19:30:241
2275545994036,3cyclictest2435529-21kworker/u8:2+events_unbound22:56:441
2275545994036,3cyclictest2275749-21kworker/u8:3+events_unbound19:15:251
2275542994033,5cyclictest2590985-21latency_hist00:15:000
2275550993932,5cyclictest2445117-21ntpq21:50:213
2275546993934,3cyclictest2575552-21kworker/u8:1+flush-259:000:10:152
2275550993835,2cyclictest2311521-21kworker/u8:2+events_unbound19:55:223
2275550993834,3cyclictest2518770-21kworker/u8:5+flush-259:023:05:213
2275550993834,3cyclictest2435529-21kworker/u8:2+events_unbound23:36:133
2275550993834,3cyclictest2363106-21kworker/u8:2+events_unbound21:12:453
2275545993834,2cyclictest1564019-21kworker/u8:2+flush-259:019:10:441
2275550993733,3cyclictest2414986-21kworker/u8:1+events_unbound22:49:453
2275550993733,2cyclictest2414986-21kworker/u8:1+flush-259:021:30:173
2275546993731,4cyclictest2451084-21latency_hist22:00:002
2275546993729,5cyclictest2595299-21ntpq00:15:222
2275545993734,2cyclictest2414986-21kworker/u8:1+events_unbound22:46:031
2275545993733,3cyclictest2456879-21kworker/u8:0+events_unbound22:22:441
2275545993733,3cyclictest2435529-21kworker/u8:2+events_unbound22:06:441
2275545993733,3cyclictest2414986-21kworker/u8:1+events_unbound22:10:141
2275542993731,4cyclictest2379214-21apt-get20:50:120
2275542993731,4cyclictest2357331-21kworker/u8:0+flush-259:021:20:250
2275550993633,2cyclictest2575552-21kworker/u8:1+events_unbound00:30:013
2275550993633,2cyclictest2575552-21kworker/u8:1+events_unbound00:30:003
2275550993633,2cyclictest2456879-21kworker/u8:0+flush-259:022:20:233
2275550993632,3cyclictest2435529-21kworker/u8:2+events_unbound22:54:453
2275550993632,3cyclictest2425360-21kworker/u8:3+flush-259:021:35:213
2275550993632,3cyclictest2357331-21kworker/u8:0+events_unbound20:29:443
2275545993633,2cyclictest2456879-21kworker/u8:0+events_unbound22:15:221
2275545993632,3cyclictest2591213-21kworker/u8:0+events_unbound00:28:451
2275545993632,3cyclictest2591213-21kworker/u8:0+events_unbound00:28:441
2275545993632,3cyclictest2357331-21kworker/u8:0+flush-259:020:50:421
2275550993532,2cyclictest2357331-21kworker/u8:0+events_unbound21:07:443
2275545993531,3cyclictest2518770-21kworker/u8:5+events_unbound23:31:121
2275542993531,3cyclictest2435529-21kworker/u8:2+flush-259:022:10:260
2275550993431,2cyclictest2456879-21kworker/u8:0+flush-259:022:25:213
2275550993431,2cyclictest2456879-21kworker/u8:0+flush-259:022:15:203
2275550993431,2cyclictest2363106-21kworker/u8:2+events_unbound20:56:453
2275550993431,2cyclictest2301559-21kworker/u8:1+events_unbound20:31:453
2275550993430,3cyclictest2503046-21kworker/u8:0+events_unbound23:02:453
2275550993430,3cyclictest2275749-21kworker/u8:3+events_unbound19:27:443
2275545993428,4cyclictest252-21systemd-journal22:32:431
2275550993331,2cyclictest2373414-21kworker/u8:3+events_unbound20:47:453
2275550993330,2cyclictest2518770-21kworker/u8:5+events_unbound23:18:443
2275550993330,2cyclictest2435529-21kworker/u8:2+flush-259:000:00:003
2275550993329,3cyclictest2357331-21kworker/u8:0+events_unbound20:51:453
2275546993326,5cyclictest252-21systemd-journal21:15:432
2275545993330,3cyclictest2357331-21kworker/u8:0+events_unbound21:10:441
2275545993330,2cyclictest2591213-21kworker/u8:0+flush-259:000:15:211
2275545993330,2cyclictest2575552-21kworker/u8:1+events_unbound00:38:261
2275545993330,2cyclictest2435529-21kworker/u8:2+events_unbound23:21:111
2275545993329,3cyclictest2607062-21kworker/u8:2+events_unbound00:32:441
2275545993329,3cyclictest2414986-21kworker/u8:1+events_unbound22:00:171
2275550993228,3cyclictest2306728-21/usr/sbin/munin19:40:133
2275546993225,5cyclictest252-21systemd-journal23:01:052
2275545993229,2cyclictest2414986-21kworker/u8:1+events_unbound21:30:501
2275545993229,2cyclictest2357331-21kworker/u8:0+events_unbound21:21:451
2275545993228,3cyclictest2301559-21kworker/u8:1+events_unbound20:05:231
2275542993228,3cyclictest2435529-21kworker/u8:2+flush-259:021:50:140
2275542993227,4cyclictest2472140-21kworker/u8:3+flush-259:022:35:220
2275550993129,2cyclictest2357331-21kworker/u8:0+flush-259:021:20:243
2275550993128,2cyclictest2301559-21kworker/u8:1+events_unbound20:36:443
2275550993127,3cyclictest2414986-21kworker/u8:1+events_unbound22:01:443
2275545993128,2cyclictest2435529-21kworker/u8:2+events_unbound22:27:451
2275545993127,3cyclictest2518770-21kworker/u8:5+events_unbound23:45:131
2275545993125,4cyclictest252-21systemd-journal20:05:001
2275542993127,3cyclictest2357331-21kworker/u8:0+flush-259:021:05:250
2275550993027,2cyclictest2472140-21kworker/u8:3+events_unbound22:42:433
2275550993027,2cyclictest2414986-21kworker/u8:1+events_unbound22:05:243
2275550993024,2cyclictest1564019-21kworker/u8:2+events_unbound19:10:223
2275545993027,3cyclictest2363106-21kworker/u8:2+flush-259:021:15:191
2275545993027,2cyclictest2575552-21kworker/u8:1+flush-259:000:20:211
2275545993027,2cyclictest2575552-21kworker/u8:1+flush-259:000:20:211
2275545993026,3cyclictest2518770-21kworker/u8:5+events_unbound23:58:431
2275545993026,3cyclictest2301559-21kworker/u8:1+events_unbound20:38:431
2275542993023,5cyclictest2563894-21ntpq23:45:210
2275542993022,5cyclictest2611084-21ntpq00:30:200
2275550992926,2cyclictest2575552-21kworker/u8:1+events_unbound00:38:453
2275546992923,4cyclictest2617423-21cat00:40:012
2275545992925,3cyclictest2275749-21kworker/u8:3+events_unbound19:28:431
2275545992923,4cyclictest252-21systemd-journal21:02:441
2275542992926,2cyclictest2394418-21kworker/u8:3+events_unbound21:05:010
227554299290,27cyclictest1-21systemd19:22:430
2275550992825,3cyclictest2518770-21kworker/u8:5+events_unbound23:41:133
2275550992825,2cyclictest2472140-21kworker/u8:3+flush-259:022:35:163
2275546992822,4cyclictest2420175-21latency_hist21:30:002
2275545992825,2cyclictest2503046-21kworker/u8:0+events_unbound22:52:441
2275545992824,3cyclictest2472140-21kworker/u8:3+events_unbound22:38:451
2275545992823,3cyclictest2575779-21/usr/sbin/munin00:00:151
2275550992724,2cyclictest2301559-21kworker/u8:1+flush-259:019:35:163
2275550992724,2cyclictest2285470-21kworker/u8:0+events_unbound19:20:273
2275550992722,2cyclictest2357331-21kworker/u8:0+events_unbound21:40:223
2275546992723,3cyclictest2357331-21kworker/u8:0+flush-259:021:35:002
2275546992723,3cyclictest2285602-21kworker/u8:4+flush-259:019:40:162
2275545992724,3cyclictest2311521-21kworker/u8:2+events_unbound20:20:281
2275542992723,3cyclictest2311521-21kworker/u8:2+events_unbound20:25:220
2275542992721,4cyclictest2372519-21ntpq20:40:210
2275550992622,3cyclictest2342779-21/usr/sbin/munin20:15:153
2275545992622,3cyclictest2446207-21apt-get21:55:111
2275542992622,3cyclictest2435529-21kworker/u8:2+events_unbound22:40:240
227554299261,23cyclictest2617294-21munin-run00:40:000
2275550992522,2cyclictest2607062-21kworker/u8:2+events_unbound00:31:433
2275550992522,2cyclictest2363106-21kworker/u8:2+flush-259:021:00:443
2275545992522,2cyclictest2357331-21kworker/u8:0+flush-259:021:40:191
2275542992522,2cyclictest2285602-21kworker/u8:4+flush-259:019:35:190
2275542992519,4cyclictest252-21systemd-journal20:46:430
2275550992421,2cyclictest2311521-21kworker/u8:2+events_unbound19:50:283
227554699241,21cyclictest1-21systemd23:30:002
2275545992419,4cyclictest2373414-21kworker/u8:3+flush-259:020:45:111
2275542992413,10cyclictest2363106-21kworker/u8:2+flush-259:021:15:230
2275550992320,2cyclictest2301559-21kworker/u8:1+events_unbound20:00:173
2275550992320,2cyclictest2301559-21kworker/u8:1+events_unbound19:47:443
2275550992320,2cyclictest2301559-21kworker/u8:1+events_unbound19:47:433
2275550992319,3cyclictest2456879-21kworker/u8:0+flush-259:022:35:003
2275545992321,2cyclictest2425360-21kworker/u8:3+events_unbound21:35:251
2275545992320,2cyclictest2518770-21kworker/u8:5+events_unbound00:07:291
2275545992320,2cyclictest2435529-21kworker/u8:2+events_unbound23:40:211
2275545992320,2cyclictest2414986-21kworker/u8:1+flush-259:023:05:441
2275545992319,3cyclictest2518770-21kworker/u8:5+flush-259:023:10:441
2275542992320,2cyclictest2435529-21kworker/u8:2+events_unbound23:15:230
2275542992320,2cyclictest2414986-21kworker/u8:1+flush-259:021:25:200
2275542992319,3cyclictest2518770-21kworker/u8:5+events_unbound23:25:210
2275542992319,3cyclictest2435529-21kworker/u8:2+flush-259:023:40:220
2275542992319,3cyclictest2435529-21kworker/u8:2+events_unbound23:35:200
2275542992319,3cyclictest2301559-21kworker/u8:1+events_unbound19:45:210
2275542992319,3cyclictest2301559-21kworker/u8:1+events_unbound19:45:200
2275542992319,3cyclictest2285602-21kworker/u8:4+flush-259:019:40:200
2275550992219,2cyclictest2363106-21kworker/u8:2+events_unbound21:15:183
2275546992217,3cyclictest252-21systemd-journal23:34:592
2275545992220,2cyclictest2285470-21kworker/u8:0+events_unbound19:45:291
2275545992220,2cyclictest2285470-21kworker/u8:0+events_unbound19:45:291
2275545992219,2cyclictest2435529-21kworker/u8:2+flush-259:022:40:281
2275542992218,3cyclictest2591213-21kworker/u8:0+events_unbound00:15:210
2275542992218,3cyclictest2357331-21kworker/u8:0+flush-259:020:35:160
2275550992118,2cyclictest2301559-21kworker/u8:1+flush-259:020:40:233
2275550992116,4cyclictest2584932-21ntpq00:05:223
227554699215,4cyclictest2570686-21https23:55:132
2275545992119,1cyclictest2311521-21kworker/u8:2+events_unbound19:50:221
2275545992117,3cyclictest2357331-21kworker/u8:0+flush-259:021:05:211
2275542992116,4cyclictest2456879-21kworker/u8:0+flush-259:022:30:140
2275550992017,3cyclictest2435529-21kworker/u8:2+events_unbound23:46:523
2275546992016,3cyclictest2275749-21kworker/u8:3+events_unbound19:15:192
2275546992013,4cyclictest252-21systemd-journal22:45:432
2275542992017,2cyclictest2275784-21apt-get19:10:120
2275542992016,3cyclictest2435529-21kworker/u8:2+events_unbound22:05:220
2275542992016,3cyclictest2301559-21kworker/u8:1+events_unbound20:30:230
2275550991915,3cyclictest2575552-21kworker/u8:1+events_unbound00:10:113
2275550991915,3cyclictest2301316-21cron19:35:003
227554699199,6cyclictest2471833-21sshd22:16:432
227554699190,10cyclictest2533955-21munin-plugin-st23:20:002
2275545991917,1cyclictest2518770-21kworker/u8:5+events_unbound23:50:171
2275545991917,1cyclictest2342528-21cron20:14:591
2275545991916,2cyclictest2435529-21kworker/u8:2+flush-259:023:40:001
2275545991915,3cyclictest2518770-21kworker/u8:5+events_unbound23:25:281
2275545991915,3cyclictest2435529-21kworker/u8:2+flush-259:021:50:521
2275545991915,3cyclictest2311521-21kworker/u8:2+events_unbound20:15:381
2275542991916,2cyclictest2414986-21kworker/u8:1+events_unbound22:45:230
2275550991815,3cyclictest2591213-21kworker/u8:0+flush-259:000:15:243
2275550991815,2cyclictest2575552-21kworker/u8:1+events_unbound00:00:123
2275546991814,3cyclictest2575552-21kworker/u8:1+flush-259:000:30:132
2275546991814,2cyclictest0-21swapper/222:20:162
2275545991816,1cyclictest2586083-21apt-get00:10:101
2275545991815,2cyclictest2301559-21kworker/u8:1+flush-259:020:25:261
2275542991814,3cyclictest2403833-21ntp_kernel_pll_21:10:220
2275542991814,3cyclictest2311521-21kworker/u8:2+flush-259:019:55:260
2275550991714,2cyclictest2503046-21kworker/u8:0+events_unbound22:55:243
2275550991714,2cyclictest2440820-21kworker/u8:0+flush-259:021:55:193
2275546991714,2cyclictest2456539-21apt-key22:05:002
2275546991711,4cyclictest2378712-21cp20:50:012
2275546991710,5cyclictest2384308-21/usr/sbin/munin20:55:172
2275546991710,5cyclictest2321031-21ntpq19:50:212
227554699170,5cyclictest2434801-21gcc21:40:222
2275542991714,2cyclictest60350irq/132-enp0s29f1:rx-020:55:180
2275542991714,2cyclictest2311521-21kworker/u8:2+events_unbound20:15:210
2275550991614,2cyclictest2311521-21kworker/u8:2+flush-259:020:20:193
227554699165,3cyclictest2425570-21chrt21:35:092
227554699164,3cyclictest2109693-21snmpd22:14:372
2275546991612,3cyclictest2575552-21kworker/u8:1+flush-259:000:20:162
2275546991612,3cyclictest2575552-21kworker/u8:1+flush-259:000:20:162
2275545991613,2cyclictest2435529-21kworker/u8:2+events_unbound23:15:171
2275545991613,2cyclictest2301559-21kworker/u8:1+flush-259:020:40:261
227554299169,5cyclictest2600778-21ntpq00:20:210
227554299169,5cyclictest2600778-21ntpq00:20:200
2275542991614,1cyclictest60350irq/132-enp0s29f1:rx-000:28:120
2275542991614,1cyclictest60350irq/132-enp0s29f1:rx-000:28:120
2275542991613,2cyclictest60350irq/132-enp0s29f1:rx-023:59:050
2275542991613,2cyclictest60350irq/132-enp0s29f1:rx-019:25:130
2275542991613,2cyclictest2327122-21kworker/u8:3+events_unbound20:00:220
2275550991513,2cyclictest2414986-21kworker/u8:1+events_unbound21:29:453
227554699158,5cyclictest2559225-21sed23:40:242
227554699157,6cyclictest2347818-21cat20:20:012
227554699157,5cyclictest2414958-21latency_hist21:25:002
227554699156,7cyclictest2276069-21fw_conntrack19:10:142
227554699151,12cyclictest1-21systemd22:53:432
2275545991512,2cyclictest2301559-21kworker/u8:1+events_unbound19:35:231
227554299158,5cyclictest252-21systemd-journal22:25:000
2275542991513,1cyclictest60350irq/132-enp0s29f1:rx-022:50:180
2275542991513,1cyclictest60350irq/132-enp0s29f1:rx-021:45:240
2275542991513,1cyclictest60350irq/132-enp0s29f1:rx-021:40:200
2275542991513,1cyclictest60350irq/132-enp0s29f1:rx-021:35:190
2275542991513,1cyclictest60350irq/132-enp0s29f1:rx-020:10:110
2275542991513,1cyclictest60350irq/132-enp0s29f1:rx-019:16:190
2275542991512,2cyclictest60350irq/132-enp0s29f1:rx-022:17:090
2275542991511,3cyclictest252-21systemd-journal22:25:560
2275542991511,3cyclictest2311521-21kworker/u8:2+events_unbound20:20:210
2275542991510,4cyclictest2332478-21/usr/sbin/munin20:05:140
2275550991412,2cyclictest2435529-21kworker/u8:2+flush-259:023:20:193
227554699149,4cyclictest2481960-21grep22:25:242
227554699149,4cyclictest2300853-21unixbench_multi19:30:252
227554699149,3cyclictest2456758-21cat22:05:012
227554699149,3cyclictest2353209-21tr20:25:122
227554699149,3cyclictest2327783-21grep20:00:162
227554699149,3cyclictest0-21swapper/200:25:142
227554699149,3cyclictest0-21swapper/200:25:132
227554699148,4cyclictest2399147-21head21:05:242
227554699148,4cyclictest2358321-21cpuspeed_turbos20:30:132
227554699148,4cyclictest2295262-21netstat19:25:202
227554699147,5cyclictest2497262-21timerandwakeup22:40:262
227554699147,4cyclictest2364035-21sed20:35:152
227554699147,4cyclictest2275673-21ls19:10:002
227554699146,6cyclictest2352865-21latency_hist20:25:012
227554699146,5cyclictest2565266-21apt-key23:50:012
227554699146,5cyclictest2311946-21grep19:45:132
227554699146,5cyclictest2311946-21grep19:45:122
2275546991411,2cyclictest2522821-21ntp_offset23:05:202
2275546991411,1cyclictest0-21swapper/221:10:172
2275546991411,1cyclictest0-21swapper/200:05:002
2275542991412,1cyclictest60350irq/132-enp0s29f1:rx-023:07:460
2275542991412,1cyclictest60350irq/132-enp0s29f1:rx-022:00:210
2275542991412,1cyclictest60350irq/132-enp0s29f1:rx-021:57:440
2275542991412,1cyclictest60350irq/132-enp0s29f1:rx-021:30:130
2275542991411,2cyclictest60350irq/132-enp0s29f1:rx-023:30:130
2275542991411,2cyclictest60350irq/132-enp0s29f1:rx-019:30:210
2275542991411,2cyclictest2508897-21irqcore22:55:160
227554699137,4cyclictest2322395-21/usr/sbin/munin19:55:152
227554699137,3cyclictest2291244-21sed19:25:002
227554699136,6cyclictest2488883-21grep22:35:172
227554699136,5cyclictest2552489-21grep23:35:182
2275545991311,2cyclictest2435529-21kworker/u8:2+flush-259:023:00:241
227554299139,3cyclictest2435529-21kworker/u8:2+events_unbound23:00:160
2275542991311,1cyclictest60350irq/132-enp0s29f1:rx-000:05:160
2275542991310,2cyclictest60350irq/132-enp0s29f1:rx-023:50:120
2275542991210,1cyclictest60350irq/132-enp0s29f1:rx-019:50:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional