You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-27 - 07:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sat Dec 27, 2025 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2269789994036,3cyclictest252-21systemd-journal00:24:591
2269789994034,4cyclictest2586066-21latency_hist00:15:011
2269786994035,3cyclictest2445161-21latency_hist22:00:010
2269792993531,3cyclictest2429707-21kworker/u8:0+flush-259:022:00:232
2269792993526,6cyclictest2304899-21ntpq19:40:222
2269789993530,4cyclictest2428766-21ntpq21:40:181
2269800993429,4cyclictest2523479-21kworker/u8:0+flush-259:023:15:173
2269792993430,3cyclictest2383321-21kworker/u8:4+flush-259:021:05:182
2269789993430,3cyclictest2290342-21kworker/u8:2+flush-259:020:40:001
2269800993227,3cyclictest2517904-21latency_hist23:10:003
2269792993228,3cyclictest2523479-21kworker/u8:0+flush-259:023:25:242
2269792993225,5cyclictest2471416-21latency_hist22:25:002
2269800993126,3cyclictest252-21systemd-journal19:38:183
2269792993127,3cyclictest2326371-21kworker/u8:1+flush-259:020:10:192
2269786993127,3cyclictest2601723-21apt-get00:30:130
2269792993027,2cyclictest2290342-21kworker/u8:2+flush-259:019:30:232
2269792993026,3cyclictest2570372-21kworker/u8:1+flush-259:000:05:232
2269792993024,4cyclictest2337279-21cat20:15:012
2269800992924,3cyclictest252-21systemd-journal23:46:433
2269789992923,4cyclictest2305752-21latency_hist19:45:001
2269800992722,4cyclictest252-21systemd-journal00:00:013
2269792992723,3cyclictest2290342-21kworker/u8:2+flush-259:019:35:162
2269792992722,3cyclictest2511730-21kworker/u8:1+flush-259:023:40:162
2269800992622,3cyclictest2466339-21base6422:20:003
2269800992622,3cyclictest2383321-21kworker/u8:4+flush-259:021:40:153
2269792992622,3cyclictest2316048-21kworker/u8:0+flush-259:019:55:182
2269789992623,2cyclictest2295717-21kworker/u8:3+events_unbound19:55:001
2269786992622,3cyclictest2523479-21kworker/u8:0+flush-259:023:55:250
2269786992622,3cyclictest2326371-21kworker/u8:1+flush-259:020:15:170
2269800992521,3cyclictest2383321-21kworker/u8:4+flush-259:021:10:233
2269792992521,3cyclictest2570372-21kworker/u8:1+events_unbound00:10:252
2269792992520,3cyclictest2570372-21kworker/u8:1+flush-259:000:00:192
2269789992521,3cyclictest2326371-21kworker/u8:1+flush-259:020:30:261
2269786992520,4cyclictest2517043-21ntpq23:05:200
2269800992419,4cyclictest2363033-21apt-get20:40:133
2269800992418,4cyclictest252-21systemd-journal21:50:003
2269792992421,2cyclictest2460915-21kworker/u8:0+flush-259:022:40:162
2269792992420,3cyclictest2326371-21kworker/u8:1+flush-259:020:25:242
2269792992418,4cyclictest2398777-21latency_hist21:15:012
2269789992419,3cyclictest2559889-21latency_hist23:50:011
2269792992320,2cyclictest2549603-21kworker/u8:2+flush-259:023:50:242
2269792992317,4cyclictest2429707-21kworker/u8:0+flush-259:022:05:192
2269792992317,4cyclictest2310922-21latency_hist19:50:002
2269789992321,2cyclictest2523479-21kworker/u8:0+flush-259:023:40:221
2269789992319,3cyclictest2560146-21apt-get23:50:131
2269789992316,5cyclictest2610914-21ntpq00:35:191
2269786992320,2cyclictest2575811-21kworker/u8:2+flush-259:000:15:240
2269792992219,2cyclictest2383321-21kworker/u8:4+events_unbound21:45:242
2269792992218,2cyclictest2557308-21kthreadcore23:45:182
2269789992219,2cyclictest2465854-21gpgv22:19:591
2269786992216,4cyclictest252-21systemd-journal19:35:000
2269792992118,2cyclictest2533773-21munin-plugin-st23:25:012
2269792992118,2cyclictest2331520-21munin-run20:10:002
2269792992116,4cyclictest2383321-21kworker/u8:4+events_unbound21:15:122
2269792992115,4cyclictest2383321-21kworker/u8:4+flush-259:021:40:152
2269789992118,2cyclictest2326371-21kworker/u8:1+flush-259:020:15:261
2269789992117,3cyclictest2492327-21apt-get22:45:151
2269786992117,3cyclictest2269757-21kworker/u8:0+flush-259:019:20:250
2269786992115,4cyclictest2548658-21ntpq23:35:180
2269800992017,2cyclictest2492290-21kworker/u8:3+flush-259:022:50:233
2269800992017,2cyclictest2492290-21kworker/u8:3+flush-259:022:50:223
2269800992016,3cyclictest2523479-21kworker/u8:0+flush-259:023:40:253
2269792992016,3cyclictest2378409-21kworker/u8:0+flush-259:020:55:192
2269792992015,3cyclictest2518093-21apt-key23:10:002
2269792992014,5cyclictest2586332-21apt-get00:15:132
2269792992014,4cyclictest2570611-21latency_hist00:00:002
2269792992013,5cyclictest2476815-21/usr/sbin/munin22:30:172
2269792992012,6cyclictest252-21systemd-journal22:39:592
2269789992016,3cyclictest2497494-21/usr/sbin/munin22:50:151
2269789992016,3cyclictest2497494-21/usr/sbin/munin22:50:141
2269786992016,3cyclictest2523479-21kworker/u8:0+flush-259:023:45:260
2269786992014,4cyclictest2414439-21apt-get21:30:130
2269800991916,2cyclictest2388463-21cron21:05:003
2269792991916,2cyclictest2511730-21kworker/u8:1+events_unbound23:00:222
2269792991915,3cyclictest2511730-21kworker/u8:1+events_unbound23:30:242
2269792991915,3cyclictest2383321-21kworker/u8:4+flush-259:021:30:252
2269792991914,4cyclictest2511730-21kworker/u8:1+flush-259:023:10:192
2269792991912,5cyclictest2611854-21latency_hist00:40:012
2269786991915,3cyclictest2575795-21latency_hist00:05:010
2269786991915,2cyclictest2554775-21munin-plugin-st23:45:000
2269786991910,2cyclictest2560554-21irqcore23:50:170
2269800991815,2cyclictest0-21swapper/319:50:013
2269800991814,3cyclictest2460915-21kworker/u8:0+flush-259:022:30:243
2269800991812,4cyclictest2403925-21latency_hist21:20:013
2269792991815,2cyclictest2419544-21kworker/u8:2+flush-259:022:10:142
2269792991814,3cyclictest2559924-21kworker/u8:3+flush-259:000:20:242
2269792991814,3cyclictest2460915-21kworker/u8:0+flush-259:022:55:252
2269792991814,2cyclictest0-21swapper/220:45:002
2269789991815,2cyclictest2393631-21munin-plugin-st21:10:011
2269786991814,3cyclictest252-21systemd-journal22:45:010
2269786991814,3cyclictest2409345-21apt-get21:25:100
2269786991814,3cyclictest2337085-21gpgv20:15:000
2269800991714,2cyclictest2580936-21grep00:10:013
2269800991713,3cyclictest2575811-21kworker/u8:2+flush-259:000:15:273
2269792991714,2cyclictest2499836-21kthreadcore22:50:172
2269792991714,2cyclictest2499836-21kthreadcore22:50:172
2269792991713,3cyclictest2326371-21kworker/u8:1+flush-259:020:35:232
2269792991712,4cyclictest2352502-21kworker/u8:3+flush-259:021:20:192
2269792991712,3cyclictest2352502-21kworker/u8:3+flush-259:020:50:212
2269789991713,3cyclictest2460915-21kworker/u8:0+flush-259:022:25:231
2269789991711,4cyclictest252-21systemd-journal20:50:001
2269786991714,2cyclictest60350irq/132-enp0s29f1:rx-023:25:230
2269786991714,2cyclictest60350irq/132-enp0s29f1:rx-019:55:210
2269786991714,2cyclictest2419544-21kworker/u8:2+flush-259:022:35:170
2269786991712,4cyclictest2326779-21diskstats20:05:120
2269786991712,3cyclictest2347344-21sh20:25:000
226978699170,14cyclictest509-21rs:main0
2269800991614,1cyclictest2352472-21idleruntime-cro20:30:003
2269800991613,2cyclictest2393148-21unixbench_multi21:05:263
2269800991612,3cyclictest2511730-21kworker/u8:1+flush-259:023:20:153
226979299169,5cyclictest2356710-21ntpq20:30:222
2269792991612,3cyclictest2383321-21kworker/u8:4+flush-259:021:00:242
2269792991612,3cyclictest2295717-21kworker/u8:3+flush-259:020:00:242
2269792991610,4cyclictest2471652-21/usr/sbin/munin22:25:152
2269789991612,3cyclictest2316048-21kworker/u8:0+flush-259:019:55:221
2269786991614,1cyclictest60350irq/132-enp0s29f1:rx-020:50:140
2269786991614,1cyclictest60350irq/132-enp0s29f1:rx-019:50:230
2269786991613,2cyclictest60350irq/132-enp0s29f1:rx-019:45:180
2269786991613,2cyclictest2352502-21kworker/u8:3+flush-259:021:25:000
2269786991612,3cyclictest2460915-21kworker/u8:0+flush-259:022:55:220
2269786991612,3cyclictest2450535-21kworker/u8:3+flush-259:022:16:280
2269786991612,3cyclictest2326371-21kworker/u8:1+flush-259:020:35:260
2269800991512,2cyclictest2357606-21cron20:35:003
2269800991512,2cyclictest2320793-21unixbench_singl19:55:253
2269800991511,3cyclictest2455510-21latency_hist22:10:013
2269800991511,3cyclictest2408774-21journal-offline21:21:243
226979299159,4cyclictest2596011-21kworker/u8:1+flush-259:000:25:042
2269792991512,2cyclictest2347357-21turbostat.cron20:25:002
2269792991511,3cyclictest2523479-21kworker/u8:0+events_unbound23:15:272
2269789991512,2cyclictest2523479-21kworker/u8:0+flush-259:023:55:231
2269789991512,2cyclictest2344771-21kthreadcore20:20:171
2269789991512,2cyclictest0-21swapper/120:10:001
2269789991511,3cyclictest252-21systemd-journal21:39:171
2269786991514,1cyclictest60350irq/132-enp0s29f1:rx-021:35:170
2269786991513,2cyclictest61150irq/133-enp0s29f1:tx-022:45:220
2269786991513,1cyclictest60350irq/132-enp0s29f1:rx-021:40:220
2269786991513,1cyclictest60350irq/132-enp0s29f1:rx-021:05:240
2269786991513,1cyclictest60350irq/132-enp0s29f1:rx-020:40:160
2269786991512,2cyclictest61150irq/133-enp0s29f1:tx-000:10:150
2269786991512,2cyclictest60350irq/132-enp0s29f1:rx-020:45:180
2269786991512,2cyclictest60350irq/132-enp0s29f1:rx-020:00:150
2269786991512,2cyclictest60350irq/132-enp0s29f1:rx-019:10:130
2269786991511,3cyclictest2559924-21kworker/u8:3+events_unbound00:20:160
226980099149,3cyclictest2347350-21latency_hist20:25:003
2269800991411,2cyclictest2511730-21kworker/u8:1+flush-259:023:00:233
2269800991411,2cyclictest2440490-21fschecks_count21:55:133
2269800991411,2cyclictest2419544-21kworker/u8:2+events_unbound22:10:183
2269800991411,2cyclictest2259512-21kworker/u8:1+flush-259:019:20:003
2269800991411,2cyclictest0-21swapper/300:25:003
2269800991410,3cyclictest509-21rs:main3
226979299149,3cyclictest2429707-21kworker/u8:0+events_unbound21:55:172
2269792991410,3cyclictest2476804-21kworker/u8:1+flush-259:022:45:312
2269789991410,3cyclictest2306023-21apt-get19:45:141
226978699147,6cyclictest0-21swapper/019:40:220
2269786991413,1cyclictest60350irq/132-enp0s29f1:rx-023:20:310
2269786991413,1cyclictest60350irq/132-enp0s29f1:rx-023:12:050
2269786991413,1cyclictest60350irq/132-enp0s29f1:rx-000:36:240
2269786991412,2cyclictest60350irq/132-enp0s29f1:rx-000:28:210
2269786991412,1cyclictest60350irq/132-enp0s29f1:rx-023:15:250
2269786991412,1cyclictest60350irq/132-enp0s29f1:rx-023:04:520
2269786991412,1cyclictest60350irq/132-enp0s29f1:rx-022:30:060
2269786991412,1cyclictest60350irq/132-enp0s29f1:rx-021:45:140
2269786991412,1cyclictest60350irq/132-enp0s29f1:rx-021:10:170
2269786991412,1cyclictest60350irq/132-enp0s29f1:rx-020:55:470
2269786991412,1cyclictest60350irq/132-enp0s29f1:rx-019:37:400
2269786991411,2cyclictest60350irq/132-enp0s29f1:rx-022:20:150
2269786991411,2cyclictest60350irq/132-enp0s29f1:rx-021:05:010
2269786991411,2cyclictest60350irq/132-enp0s29f1:rx-020:32:530
2269786991411,2cyclictest2440019-21idleruntime-cro21:55:000
2269786991410,3cyclictest2419544-21kworker/u8:2+flush-259:022:10:170
226980099139,3cyclictest2378409-21kworker/u8:0+flush-259:020:55:243
226980099139,2cyclictest0-21swapper/321:40:013
2269800991311,1cyclictest0-21swapper/320:15:003
2269800991310,2cyclictest252-21systemd-journal22:01:263
226979299139,3cyclictest2383321-21kworker/u8:4+flush-259:021:27:542
226979299139,3cyclictest2305798-21kworker/u8:1+events_unbound19:50:222
226979299137,4cyclictest2285426-21/usr/sbin/munin19:25:142
2269792991310,2cyclictest2270130-21cpuspeed_turbos19:10:132
2269792991310,2cyclictest2259512-21kworker/u8:1+flush-259:019:15:132
226978999139,3cyclictest252-21systemd-journal20:58:181
226978999139,3cyclictest2517905-21apt-get23:10:011
2269789991310,2cyclictest2523833-21cpuspeed_turbos23:15:131
2269789991310,2cyclictest2414860-21sed21:30:181
2269789991310,2cyclictest2383321-21kworker/u8:4+flush-259:021:30:001
2269789991310,2cyclictest2352502-21kworker/u8:3+flush-259:020:50:261
2269789991310,2cyclictest2289901-21unixbench_singl19:25:251
2269789991310,2cyclictest2259512-21kworker/u8:1+flush-259:019:10:171
2269789991310,1cyclictest0-21swapper/100:30:001
226978699137,4cyclictest509-21rs:main0
226978699137,4cyclictest509-21rs:main0
226978699131,3cyclictest2476567-21awk22:30:000
2269786991312,1cyclictest60350irq/132-enp0s29f1:rx-000:05:150
2269786991311,1cyclictest60350irq/132-enp0s29f1:rx-022:00:260
2269786991311,1cyclictest60350irq/132-enp0s29f1:rx-021:19:150
2269786991311,1cyclictest60350irq/132-enp0s29f1:rx-020:25:150
226980099129,2cyclictest2565052-21cron23:55:013
226980099129,2cyclictest2471744-21cpuspeed_turbos22:25:143
2269800991210,1cyclictest0-21swapper/320:55:003
226979299125,5cyclictest2440035-21latency_hist21:55:022
226978999129,2cyclictest2579963-21needreboot00:05:211
226978999129,2cyclictest2502903-21fschecks_count22:55:141
226978999129,2cyclictest2460915-21kworker/u8:0+flush-259:022:20:191
226978999129,2cyclictest2295078-21unixbench_singl19:30:241
226978999129,2cyclictest2280402-21cpuspeed_turbos19:20:131
226978999128,3cyclictest2596011-21kworker/u8:1+flush-259:000:30:231
226978999125,5cyclictest2541880-21sshd23:30:181
2269789991210,1cyclictest0-21swapper/121:25:001
226978699129,1cyclictest60350irq/132-enp0s29f1:rx-019:25:130
2269786991210,1cyclictest60350irq/132-enp0s29f1:rx-022:05:250
226980099118,2cyclictest2461371-21irqcore22:15:183
226980099118,2cyclictest2439603-21unixbench_singl21:50:233
226980099118,2cyclictest2311289-21cpuspeed_turbos19:50:123
226980099118,2cyclictest2259512-21kworker/u8:1+flush-259:019:10:253
226979299118,2cyclictest2383321-21kworker/u8:4+flush-259:022:15:162
226979299117,3cyclictest2259512-21kworker/u8:1+events_unbound19:20:122
226978999119,1cyclictest0-21swapper/120:15:011
226978999119,1cyclictest0-21swapper/119:15:191
226978999118,2cyclictest2482002-21apt-get22:35:121
226978999118,2cyclictest2347675-21cpuspeed_turbos20:25:141
226978999118,2cyclictest2321897-21irqcore20:00:161
226978999116,3cyclictest2574820-21cut00:00:201
226980099108,1cyclictest2282618-21kthreadcore19:20:183
226980099107,2cyclictest2601789-21cpuspeed_turbos00:30:143
226980099106,2cyclictest252-21systemd-journal21:34:183
226978999108,1cyclictest0-21swapper/123:25:181
226978999108,1cyclictest0-21swapper/122:05:191
226978999107,2cyclictest2383782-21fschecks_time21:00:151
226978999106,2cyclictest2549570-21awk23:40:011
22698009996,2cyclictest0-21swapper/300:00:153
22697899996,2cyclictest0-21swapper/121:10:191
22697899994,4cyclictest311ktimers/120:40:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional