You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-26 - 14:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Fri Dec 26, 2025 00:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3258518994138,2cyclictest3502648-21kworker/u8:5+flush-259:023:25:173
3258518993633,2cyclictest3300420-21kworker/u8:3+flush-259:020:15:203
3258513993327,5cyclictest3476429-21cp22:40:012
3258510993227,3cyclictest3372286-21kworker/u8:1+flush-259:021:10:181
3258510993225,5cyclictest252-21systemd-journal19:12:171
3258513993127,3cyclictest3372286-21kworker/u8:1+flush-259:022:25:012
3258518993025,4cyclictest3554377-21/usr/sbin/munin23:55:143
3258518993025,4cyclictest3507815-21latency_hist23:10:013
3258513993025,3cyclictest3523470-21kworker/u8:0+flush-259:023:55:002
3258513992822,4cyclictest3279334-21/usr/sbin/munin19:30:142
3258510992823,3cyclictest3580104-21base6400:20:001
3258508992822,4cyclictest3289845-21diskstats19:40:160
3258518992724,2cyclictest3300420-21kworker/u8:3+flush-259:022:00:173
3258518992723,3cyclictest3263633-21kworker/u8:4+flush-259:019:35:293
3258518992622,3cyclictest3544046-21kworker/u8:3+flush-259:000:05:233
3258518992621,3cyclictest3544046-21kworker/u8:3+events_unbound23:50:213
3258513992621,3cyclictest252-21systemd-journal23:37:172
3258510992622,3cyclictest3300420-21kworker/u8:3+flush-259:020:30:271
3258518992522,2cyclictest3300420-21kworker/u8:3+flush-259:021:55:223
3258518992522,2cyclictest3243183-21kworker/u8:3+flush-259:019:35:003
3258518992521,3cyclictest3574748-21kworker/u8:0+flush-259:000:20:223
3258518992521,3cyclictest3336633-21kworker/u8:2+flush-259:020:35:233
3258518992520,3cyclictest252-21systemd-journal00:28:343
3258513992518,5cyclictest252-21systemd-journal21:15:002
3258508992520,3cyclictest3450826-21apt-get22:15:130
3258513992419,3cyclictest3502647-21kworker/u8:2+flush-259:023:40:212
3258510992418,4cyclictest3471147-21latency_hist22:35:001
3258510992417,4cyclictest3419229-21latency_hist21:45:001
3258508992418,4cyclictest252-21systemd-journal22:35:010
3258508992417,5cyclictest3258870-21diskstats19:10:130
3258518992317,4cyclictest3569944-21apt-get00:10:143
3258508992317,4cyclictest3331588-21latency_hist20:20:000
3258518992217,4cyclictest3263633-21kworker/u8:4+flush-259:019:20:223
3258518992217,3cyclictest3502648-21kworker/u8:5+flush-259:023:20:233
3258518992216,4cyclictest3461081-21apt-get22:25:133
3258508992217,3cyclictest3502891-21apt-get23:05:120
3258508992217,3cyclictest3372286-21kworker/u8:1+flush-259:022:25:160
3258508992215,5cyclictest3517176-21ntpq23:15:210
3258518992117,3cyclictest3377959-21kworker/u8:2+events_unbound21:05:253
3258518992117,3cyclictest3300420-21kworker/u8:3+flush-259:020:30:233
3258518992117,3cyclictest3300420-21kworker/u8:3+events_unbound21:50:223
3258518992116,3cyclictest3502647-21kworker/u8:2+flush-259:023:30:263
3258518992116,3cyclictest3466217-21/usr/sbin/munin22:30:153
3258518992115,4cyclictest252-21systemd-journal00:35:003
3258513992117,3cyclictest3300420-21kworker/u8:3+flush-259:020:20:002
3258508992118,2cyclictest252-21systemd-journal21:10:000
3258518992018,1cyclictest3471372-21kworker/u8:3+events_unbound22:40:223
3258518992016,3cyclictest3502648-21kworker/u8:5+flush-259:023:10:223
3258518992016,3cyclictest3362547-21kworker/u8:0+events_unbound21:00:173
3258518992016,3cyclictest3294538-21kworker/u8:0+events_unbound20:05:133
3258518992016,3cyclictest3294538-21kworker/u8:0+events_unbound20:05:133
3258518992016,3cyclictest3263633-21kworker/u8:4+events_unbound19:50:143
3258518992015,4cyclictest3596123-21apt-get00:35:143
3258518992015,3cyclictest3439453-21kworker/u8:0+flush-259:022:55:213
3258518992015,3cyclictest3414061-21munin-plugin-st21:40:003
3258513992017,2cyclictest3300420-21kworker/u8:3+flush-259:020:10:232
3258513992016,3cyclictest3502648-21kworker/u8:5+flush-259:023:25:222
3258513992014,4cyclictest252-21systemd-journal19:50:182
3258510992017,2cyclictest3502648-21kworker/u8:5+flush-259:023:35:211
3258508992015,3cyclictest3336633-21kworker/u8:2+flush-259:020:50:170
3258508992014,5cyclictest3408004-21ntpq21:30:220
3258518991916,2cyclictest3471372-21kworker/u8:3+events_unbound22:45:193
3258518991916,2cyclictest3263633-21kworker/u8:4+events_unbound19:20:003
3258513991915,3cyclictest3445497-21cat22:10:012
3258513991913,4cyclictest252-21systemd-journal22:09:592
3258513991913,4cyclictest252-21systemd-journal22:09:592
3258513991913,4cyclictest252-21systemd-journal20:55:012
3258510991916,2cyclictest3398573-21kworker/u8:0+flush-259:021:30:261
3258510991915,3cyclictest3502647-21kworker/u8:2+flush-259:023:31:181
3258510991914,3cyclictest3294538-21kworker/u8:0+events_unbound19:45:181
3258508991913,4cyclictest3575055-21/usr/sbin/munin00:15:150
3258518991815,2cyclictest3538744-21kworker/u8:1+flush-259:023:45:283
3258518991815,2cyclictest3502647-21kworker/u8:2+flush-259:023:40:233
3258518991814,3cyclictest3300420-21kworker/u8:3+events_unbound21:40:243
3258518991814,3cyclictest3300420-21kworker/u8:3+events_unbound20:40:233
3258513991815,2cyclictest3315874-21gpgv20:05:012
3258513991814,3cyclictest3502648-21kworker/u8:5+flush-259:023:05:242
3258513991814,3cyclictest3429519-21kworker/u8:4+events_unbound23:20:192
3258513991814,3cyclictest3377959-21kworker/u8:2+events_unbound21:05:232
3258513991814,3cyclictest3336633-21kworker/u8:2+flush-259:020:25:242
3258513991813,4cyclictest3372286-21kworker/u8:1+flush-259:021:20:002
3258513991812,4cyclictest3419475-21/usr/sbin/munin21:45:172
3258510991815,2cyclictest3528422-21sh23:30:001
3258508991814,3cyclictest60350irq/132-enp0s29f1:rx-020:40:160
3258518991713,2cyclictest3275214-21kthreadcore19:25:173
3258518991711,4cyclictest3393439-21latency_hist21:20:013
3258513991714,2cyclictest3279326-21kworker/u8:2+flush-259:019:40:232
3258513991713,3cyclictest3362547-21kworker/u8:0+events_unbound21:00:252
3258513991713,3cyclictest3300420-21kworker/u8:3+events_unbound21:50:232
3258513991712,3cyclictest3471372-21kworker/u8:3+flush-259:022:40:212
3258513991711,4cyclictest252-21systemd-journal23:49:182
3258510991714,2cyclictest3332266-21irqcore20:20:171
3258510991714,1cyclictest0-21swapper/121:00:001
3258510991713,3cyclictest3564762-21apt-get00:05:161
3258510991713,3cyclictest3544046-21kworker/u8:3+flush-259:023:50:191
3258510991713,3cyclictest3455965-21apt-get22:20:131
3258510991713,3cyclictest3439453-21kworker/u8:0+flush-259:022:35:261
3258510991713,3cyclictest3300420-21kworker/u8:3+flush-259:021:45:161
3258510991713,2cyclictest3585510-21kworker/u8:2+flush-259:000:35:191
3258510991712,3cyclictest3388511-21/usr/sbin/munin21:15:191
3258510991712,3cyclictest3300420-21kworker/u8:3+flush-259:021:55:171
3258508991713,3cyclictest3300420-21kworker/u8:3+events_unbound21:35:180
3258518991613,2cyclictest3518105-21sh23:20:003
3258518991612,2cyclictest3476334-21latency_hist22:40:003
3258513991612,3cyclictest3294538-21kworker/u8:0+flush-259:020:05:192
3258513991612,3cyclictest3294538-21kworker/u8:0+flush-259:020:05:192
3258510991614,1cyclictest0-21swapper/119:35:011
3258510991613,2cyclictest3316329-21kworker/u8:2+flush-259:020:05:241
3258510991613,2cyclictest3316329-21kworker/u8:2+flush-259:020:05:231
3258510991613,2cyclictest3243183-21kworker/u8:3+flush-259:019:40:191
3258510991611,3cyclictest3502648-21kworker/u8:5+flush-259:023:15:261
3258508991614,1cyclictest60350irq/132-enp0s29f1:rx-023:45:280
3258508991614,1cyclictest60350irq/132-enp0s29f1:rx-021:45:260
3258508991614,1cyclictest60350irq/132-enp0s29f1:rx-019:55:240
3258508991614,1cyclictest60350irq/132-enp0s29f1:rx-019:29:090
3258508991612,3cyclictest3347238-21/usr/sbin/munin20:35:220
3258518991511,3cyclictest3554367-21kworker/u8:4+flush-259:000:00:253
3258518991511,3cyclictest3372286-21kworker/u8:1+flush-259:022:20:253
3258513991512,2cyclictest3294538-21kworker/u8:0+flush-259:019:45:242
3258513991511,3cyclictest3243183-21kworker/u8:3+events_unbound19:10:012
3258513991510,4cyclictest509-21rs:main2
3258510991511,3cyclictest3574748-21kworker/u8:0+flush-259:000:30:001
3258510991510,4cyclictest3481934-21kworker/u8:1+flush-259:022:50:281
325850899159,4cyclictest3346997-21latency_hist20:35:010
3258508991514,1cyclictest60350irq/132-enp0s29f1:rx-020:57:020
3258508991514,1cyclictest60350irq/132-enp0s29f1:rx-020:25:260
3258508991514,1cyclictest60350irq/132-enp0s29f1:rx-000:39:240
3258508991513,2cyclictest60350irq/132-enp0s29f1:rx-021:10:460
3258508991513,1cyclictest60350irq/132-enp0s29f1:rx-023:55:250
3258508991513,1cyclictest60350irq/132-enp0s29f1:rx-023:50:170
3258508991513,1cyclictest60350irq/132-enp0s29f1:rx-023:30:260
3258508991513,1cyclictest60350irq/132-enp0s29f1:rx-023:10:240
3258508991513,1cyclictest60350irq/132-enp0s29f1:rx-022:55:180
3258508991513,1cyclictest60350irq/132-enp0s29f1:rx-022:51:050
3258508991513,1cyclictest60350irq/132-enp0s29f1:rx-022:10:250
3258508991513,1cyclictest60350irq/132-enp0s29f1:rx-021:55:190
3258508991513,1cyclictest60350irq/132-enp0s29f1:rx-020:20:240
3258508991513,1cyclictest60350irq/132-enp0s29f1:rx-019:33:330
3258508991512,2cyclictest60350irq/132-enp0s29f1:rx-023:24:560
3258508991512,2cyclictest60350irq/132-enp0s29f1:rx-022:00:170
3258508991512,2cyclictest60350irq/132-enp0s29f1:rx-019:37:320
3258508991512,2cyclictest3271349-21kthreadcore19:20:180
3258508991512,2cyclictest0-21swapper/019:55:000
3258518991411,2cyclictest3580281-21apt-key00:20:013
3258518991411,2cyclictest3397749-21ntp_offset21:20:233
3258518991411,2cyclictest3363937-21kthreadcore20:50:193
3258518991411,2cyclictest3300420-21kworker/u8:3+flush-259:021:10:213
3258518991410,3cyclictest3439453-21kworker/u8:0+events_unbound23:00:183
3258518991410,3cyclictest3336633-21kworker/u8:2+events_unbound20:55:173
3258518991410,3cyclictest3336633-21kworker/u8:2+events_unbound20:45:263
3258518991410,2cyclictest0-21swapper/322:10:013
3258518991410,2cyclictest0-21swapper/322:10:003
325851399147,5cyclictest252-21systemd-journal19:27:182
3258513991411,2cyclictest3544046-21kworker/u8:3+events_unbound23:55:212
3258513991411,2cyclictest3481934-21kworker/u8:1+flush-259:022:50:232
3258513991411,2cyclictest0-21swapper/200:39:592
3258513991410,3cyclictest3439453-21kworker/u8:0+flush-259:023:00:172
325851099149,4cyclictest3305636-21apt-get19:55:101
325851099149,3cyclictest3502648-21kworker/u8:5+flush-259:023:20:251
3258510991410,3cyclictest3372286-21kworker/u8:1+flush-259:022:25:221
3258510991410,3cyclictest3310281-21kworker/u8:1+flush-259:020:05:001
325850899148,4cyclictest3528422-21latency_hist23:30:010
3258508991413,1cyclictest60350irq/132-enp0s29f1:rx-022:22:190
3258508991413,1cyclictest60350irq/132-enp0s29f1:rx-020:05:130
3258508991413,1cyclictest60350irq/132-enp0s29f1:rx-020:05:120
3258508991412,2cyclictest60350irq/132-enp0s29f1:rx-021:26:120
3258508991412,2cyclictest60350irq/132-enp0s29f1:rx-000:29:170
3258508991412,1cyclictest60350irq/132-enp0s29f1:rx-022:35:130
3258508991412,1cyclictest60350irq/132-enp0s29f1:rx-021:40:260
3258508991412,1cyclictest60350irq/132-enp0s29f1:rx-021:15:170
3258508991412,1cyclictest60350irq/132-enp0s29f1:rx-019:46:360
3258508991412,1cyclictest60350irq/132-enp0s29f1:rx-000:22:060
3258508991412,1cyclictest60350irq/132-enp0s29f1:rx-000:00:130
3258508991412,1cyclictest3574818-21turbostat00:15:000
3258508991411,2cyclictest60350irq/132-enp0s29f1:rx-023:42:050
3258508991411,2cyclictest60350irq/132-enp0s29f1:rx-022:06:530
3258508991411,2cyclictest60350irq/132-enp0s29f1:rx-022:06:520
3258508991411,2cyclictest60350irq/132-enp0s29f1:rx-000:31:490
325851899139,2cyclictest0-21swapper/321:30:183
325851899131,9cyclictest3315687-21awk20:05:013
3258518991310,2cyclictest3300420-21kworker/u8:3+events_unbound21:45:143
325851399139,3cyclictest3372286-21kworker/u8:1+events_unbound21:59:592
325851399138,3cyclictest3439453-21kworker/u8:0+flush-259:022:30:172
325851399136,5cyclictest3486150-21ntpq22:45:222
3258513991311,1cyclictest3451851-21kthreadcore22:15:182
3258513991311,1cyclictest0-21swapper/200:30:002
3258513991311,1cyclictest0-21swapper/200:09:592
3258513991310,2cyclictest3336633-21kworker/u8:2+flush-259:020:45:292
3258513991310,2cyclictest3243183-21kworker/u8:3+events_unbound19:25:002
325851099139,3cyclictest3573922-21ntp_kernel_pll_00:10:241
325851099139,3cyclictest3502647-21kworker/u8:2+flush-259:023:40:271
325851099139,3cyclictest3263633-21kworker/u8:4+flush-259:019:40:001
3258510991311,1cyclictest3410200-21kthreadcore21:35:161
3258510991310,2cyclictest3441098-21kthreadcore22:05:191
3258510991310,2cyclictest3441098-21kthreadcore22:05:181
3258510991310,2cyclictest3429519-21kworker/u8:4+flush-259:022:00:241
3258510991310,2cyclictest3336633-21kworker/u8:2+flush-259:020:35:201
325850899139,3cyclictest3544046-21kworker/u8:3+flush-259:000:10:000
3258508991312,1cyclictest60350irq/132-enp0s29f1:rx-021:22:470
3258508991312,1cyclictest60350irq/132-enp0s29f1:rx-021:02:460
3258508991311,1cyclictest60350irq/132-enp0s29f1:rx-022:45:090
3258508991311,1cyclictest60350irq/132-enp0s29f1:rx-021:51:280
3258508991311,1cyclictest60350irq/132-enp0s29f1:rx-020:47:020
3258508991310,2cyclictest60350irq/132-enp0s29f1:rx-023:35:070
3258508991310,2cyclictest60350irq/132-enp0s29f1:rx-020:00:150
325851899128,3cyclictest3300420-21kworker/u8:3+events_unbound20:10:183
325851899125,4cyclictest252-21systemd-journal19:45:003
325851899120,4cyclictest3450547-21cron22:15:003
325851399129,2cyclictest3471372-21kworker/u8:3+flush-259:022:55:242
325851399129,2cyclictest3409314-21cpuspeed_turbos21:35:132
325851399129,2cyclictest3407993-21ntp_kernel_err21:30:212
325851399128,3cyclictest3502648-21kworker/u8:5+flush-259:023:15:222
325851399120,11cyclictest3512952-21awk23:15:002
325851099129,2cyclictest3362835-21cpuspeed_turbos20:50:131
325851099129,2cyclictest0-21swapper/123:15:001
325851099128,2cyclictest3403765-21awk21:30:001
325851099126,4cyclictest252-21systemd-journal21:54:181
3258510991210,1cyclictest0-21swapper/122:50:001
325850899128,3cyclictest490-21dbus-daemon23:05:010
3258508991210,1cyclictest60350irq/132-enp0s29f1:rx-019:15:200
325851899116,4cyclictest3429519-21kworker/u8:4+flush-259:022:15:203
325851399118,2cyclictest3554367-21kworker/u8:4+events_unbound00:00:222
325851099119,1cyclictest3356295-21needreboot20:40:201
325851099119,1cyclictest0-21swapper/123:05:011
325851099118,2cyclictest0-21swapper/100:00:201
325851099118,1cyclictest0-21swapper/123:05:191
325851099117,3cyclictest3523470-21kworker/u8:0+flush-259:023:55:191
325851099117,3cyclictest3263633-21kworker/u8:4+flush-259:019:20:231
325851099116,3cyclictest252-21systemd-journal21:04:191
325850899116,4cyclictest252-21systemd-journal20:15:000
325851899106,2cyclictest3263610-21latency_hist19:15:013
325851399107,2cyclictest3574748-21kworker/u8:0+flush-259:000:15:252
325851399106,2cyclictest3351244-21ntpq20:35:242
325851399100,2cyclictest3372803-21latency_hist21:00:012
325851099107,2cyclictest3294538-21kworker/u8:0+flush-259:020:30:001
325851099104,4cyclictest252-21systemd-journal20:45:171
325851099100,2cyclictest3268793-21latency_hist19:20:011
32585139996,2cyclictest401ktimers/222:00:202
32585109986,1cyclictest0-21swapper/100:30:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional