You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-13 - 23:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Thu Nov 13, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
256066995552,2cyclictest458558-21kworker/u8:1+events_unbound10:39:551
256064994643,2cyclictest458558-21kworker/u8:1+flush-259:010:36:270
256069994541,3cyclictest307743-21kworker/u8:3+flush-259:008:35:002
256069994239,2cyclictest286946-21kworker/u8:1+events_unbound07:50:152
256064994236,4cyclictest463718-21latency_hist10:30:010
256066994138,2cyclictest452541-21kworker/u8:2+flush-259:010:25:161
256069994036,3cyclictest452541-21kworker/u8:2+events_unbound10:35:002
256066993937,1cyclictest348967-21fwupd08:38:371
256066993831,3cyclictest317794-21kworker/u8:0+flush-259:008:30:161
256064993733,3cyclictest516253-21kworker/u8:2+flush-259:011:35:250
256066993634,2cyclictest533223-21kworker/u8:4+events_unbound11:50:161
256066993532,2cyclictest569549-21kworker/u8:3+flush-259:012:25:161
256069993330,2cyclictest490927-21kworker/u8:2+events_unbound11:10:012
256066993330,2cyclictest558953-21kworker/u8:0+events_unbound12:00:151
256066993330,2cyclictest516252-21kworker/u8:1+flush-259:012:10:171
256066993330,2cyclictest375515-21kworker/u8:2+events_unbound09:05:161
256066993229,2cyclictest317794-21kworker/u8:0+events_unbound08:15:151
256066993226,2cyclictest490927-21kworker/u8:2+events_unbound11:00:151
256064993229,2cyclictest375515-21kworker/u8:2+events_unbound09:10:240
256064993229,2cyclictest307743-21kworker/u8:3+flush-259:008:10:240
256069993128,2cyclictest395643-21kworker/u8:1+flush-259:009:30:152
256069993127,3cyclictest458558-21kworker/u8:1+flush-259:010:40:242
256069993125,4cyclictest448173-21latency_hist10:15:012
256066993128,2cyclictest490927-21kworker/u8:2+events_unbound10:55:201
256066993128,2cyclictest317794-21kworker/u8:0+flush-259:008:45:181
256074993024,5cyclictest475136-21apt-get10:39:583
256069993027,2cyclictest369733-21kworker/u8:0+events_unbound10:10:002
256074992924,3cyclictest452541-21kworker/u8:2+flush-259:010:30:183
256069992926,2cyclictest317794-21kworker/u8:0+events_unbound08:40:162
256069992926,2cyclictest203158-21kworker/u8:1+flush-259:007:10:172
256069992925,2cyclictest223473-21kworker/u8:3+flush-259:007:25:152
256066992926,2cyclictest223473-21kworker/u8:3+events_unbound07:40:151
256066992926,2cyclictest223473-21kworker/u8:3+events_unbound07:35:161
256066992926,2cyclictest223473-21kworker/u8:3+events_unbound07:25:131
256074992825,2cyclictest516252-21kworker/u8:1+flush-259:012:35:003
256069992826,2cyclictest473660-21kworker/u8:3+flush-259:010:37:062
256069992825,2cyclictest490927-21kworker/u8:2+events_unbound11:04:022
256069992825,2cyclictest307743-21kworker/u8:3+flush-259:008:25:262
256066992825,2cyclictest286946-21kworker/u8:1+flush-259:007:55:241
256064992821,6cyclictest558801-21latency_hist12:00:000
256074992722,3cyclictest533419-21diskstats11:35:143
256069992724,2cyclictest395643-21kworker/u8:1+flush-259:009:25:242
256069992724,2cyclictest369733-21kworker/u8:0+events_unbound10:15:232
256066992725,2cyclictest501260-21kworker/u8:4+events_unbound11:15:151
256066992725,2cyclictest369733-21kworker/u8:0+events_unbound09:00:251
256066992725,1cyclictest480439-21kworker/u8:0+flush-259:010:50:001
256066992724,2cyclictest375515-21kworker/u8:2+flush-259:009:25:211
256064992722,4cyclictest453354-21latency_hist10:20:010
256074992622,3cyclictest317794-21kworker/u8:0+events_unbound08:45:253
256074992620,5cyclictest2698871-21systemd-journal07:50:013
256069992623,2cyclictest307743-21kworker/u8:3+flush-259:008:20:182
256066992623,2cyclictest516253-21kworker/u8:2+events_unbound11:45:241
256066992623,2cyclictest375515-21kworker/u8:2+events_unbound09:20:141
256066992623,2cyclictest307743-21kworker/u8:3+events_unbound08:40:231
256066992622,3cyclictest522504-21/usr/sbin/munin11:25:181
256064992623,2cyclictest500651-21kworker/u8:0+events_unbound11:11:380
256064992623,2cyclictest317794-21kworker/u8:0+flush-259:008:35:270
256074992519,4cyclictest2698871-21systemd-journal07:53:413
256069992522,2cyclictest516252-21kworker/u8:1+events_unbound12:15:162
256069992522,2cyclictest458558-21kworker/u8:1+events_unbound10:50:162
256066992523,1cyclictest473660-21kworker/u8:3+events_unbound10:40:151
256074992421,2cyclictest307743-21kworker/u8:3+events_unbound08:05:143
256074992420,3cyclictest516253-21kworker/u8:2+flush-259:011:25:203
256074992420,3cyclictest369733-21kworker/u8:0+flush-259:009:05:163
256069992421,2cyclictest375515-21kworker/u8:2+flush-259:009:10:202
256066992419,4cyclictest427015-21latency_hist09:55:001
256064992421,2cyclictest364255-21kworker/u8:4+flush-259:009:15:160
256064992421,2cyclictest317794-21kworker/u8:0+flush-259:008:40:210
256074992320,2cyclictest490927-21kworker/u8:2+events_unbound10:55:183
256074992319,3cyclictest533223-21kworker/u8:4+events_unbound11:40:153
256074992319,3cyclictest400720-21apt-get09:25:403
256074992317,4cyclictest2698871-21systemd-journal09:04:403
256069992320,2cyclictest516253-21kworker/u8:2+flush-259:012:05:002
256069992320,2cyclictest501260-21kworker/u8:4+events_unbound11:15:222
256069992320,2cyclictest458558-21kworker/u8:1+flush-259:010:45:172
256069992320,2cyclictest369733-21kworker/u8:0+events_unbound09:40:242
256069992320,2cyclictest223473-21kworker/u8:3+flush-259:007:35:152
256066992320,2cyclictest589787-21kworker/u8:0+events_unbound12:30:151
256066992320,2cyclictest533223-21kworker/u8:4+events_unbound11:40:231
256066992320,2cyclictest317794-21kworker/u8:0+events_unbound08:50:231
256066992320,2cyclictest286946-21kworker/u8:1+events_unbound07:45:011
256064992320,2cyclictest516829-21gpgv11:20:010
256074992219,2cyclictest501260-21kworker/u8:4+flush-259:011:20:173
256074992217,4cyclictest364255-21kworker/u8:4+flush-259:009:15:193
256069992219,2cyclictest473660-21kworker/u8:3+events_unbound11:10:232
256066992220,1cyclictest307743-21kworker/u8:3+flush-259:008:05:151
256066992219,2cyclictest432189-21kworker/u8:1+events_unbound10:10:151
256066992219,2cyclictest432189-21kworker/u8:1+events_unbound10:10:141
256066992219,2cyclictest401357-21/usr/sbin/munin09:30:231
256066992218,3cyclictest266543-21/usr/sbin/munin07:20:151
256064992219,2cyclictest369733-21kworker/u8:0+flush-259:009:35:260
256064992219,2cyclictest286946-21kworker/u8:1+events_unbound07:50:200
256064992218,3cyclictest452541-21kworker/u8:2+events_unbound10:30:190
256064992216,2cyclictest432189-21kworker/u8:1+events_unbound10:05:200
256064992215,4cyclictest2698871-21systemd-journal08:20:000
256074992119,1cyclictest533223-21kworker/u8:4+events_unbound11:50:243
256074992118,2cyclictest516252-21kworker/u8:1+events_unbound11:49:593
256074992118,2cyclictest458558-21kworker/u8:1+flush-259:010:50:173
256069992120,1cyclictest223473-21kworker/u8:3+events_unbound07:30:152
256069992118,2cyclictest307743-21kworker/u8:3+events_unbound08:05:162
256069992118,2cyclictest286946-21kworker/u8:1+flush-259:007:45:192
256069992118,2cyclictest203158-21kworker/u8:1+flush-259:007:15:252
256066992118,2cyclictest516252-21kworker/u8:1+flush-259:012:15:231
256066992118,2cyclictest516252-21kworker/u8:1+events_unbound12:05:151
256066992118,2cyclictest286946-21kworker/u8:1+events_unbound07:54:591
256064992118,2cyclictest516253-21kworker/u8:2+flush-259:012:25:000
256064992118,2cyclictest500651-21kworker/u8:0+flush-259:011:04:180
256064992118,2cyclictest426975-21kworker/u8:2+events_unbound09:55:150
256074992016,3cyclictest317794-21kworker/u8:0+flush-259:008:50:193
256069992017,2cyclictest375515-21kworker/u8:2+events_unbound09:05:262
256069992016,3cyclictest427248-21/usr/sbin/munin09:55:152
256066992017,2cyclictest501260-21kworker/u8:4+events_unbound11:20:141
256066992016,3cyclictest503-21systemd-logind08:05:001
256066992015,4cyclictest385862-21apt-get09:15:121
256074991916,2cyclictest265940-21kworker/u8:2+flush-259:007:20:203
256074991915,3cyclictest317794-21kworker/u8:0+flush-259:008:30:193
256074991915,3cyclictest307743-21kworker/u8:3+events_unbound08:20:243
256069991917,2cyclictest307743-21kworker/u8:3+events_unbound08:00:252
256069991917,1cyclictest369733-21kworker/u8:0+flush-259:009:35:152
256069991916,2cyclictest569549-21kworker/u8:3+flush-259:012:25:212
256069991916,2cyclictest375515-21kworker/u8:2+events_unbound09:20:232
256069991915,3cyclictest490927-21kworker/u8:2+events_unbound10:55:142
256066991917,2cyclictest369733-21kworker/u8:0+events_unbound09:40:141
256066991917,2cyclictest317794-21kworker/u8:0+events_unbound08:25:151
256066991916,2cyclictest516253-21kworker/u8:2+events_unbound11:30:191
256066991916,2cyclictest432189-21kworker/u8:1+events_unbound10:05:241
256064991918,1cyclictest302395-21kworker/u8:0+events_unbound07:59:430
256064991916,2cyclictest516253-21kworker/u8:2+flush-259:011:45:190
256064991916,2cyclictest516253-21kworker/u8:2+events_unbound11:30:270
256064991916,2cyclictest516252-21kworker/u8:1+flush-259:012:10:200
256064991916,2cyclictest473660-21kworker/u8:3+events_unbound11:28:410
256064991916,2cyclictest203158-21kworker/u8:1+flush-259:007:15:170
256064991915,3cyclictest369733-21kworker/u8:0+events_unbound10:00:420
256064991915,3cyclictest369733-21kworker/u8:0+events_unbound10:00:410
256064991915,3cyclictest364255-21kworker/u8:4+events_unbound08:55:180
256064991913,4cyclictest2698871-21systemd-journal11:10:000
256074991814,3cyclictest501260-21kworker/u8:4+flush-259:011:15:123
256069991816,2cyclictest307743-21kworker/u8:3+flush-259:008:10:252
256069991816,1cyclictest432189-21kworker/u8:1+flush-259:010:00:252
256069991816,1cyclictest432189-21kworker/u8:1+flush-259:010:00:242
256069991815,2cyclictest516253-21kworker/u8:2+flush-259:011:45:252
256066991816,1cyclictest569549-21kworker/u8:3+events_unbound12:20:161
256066991815,2cyclictest369733-21kworker/u8:0+flush-259:010:00:571
256066991815,2cyclictest369733-21kworker/u8:0+flush-259:010:00:571
256066991815,2cyclictest276665-21kworker/u8:2+events_unbound08:13:431
256066991815,2cyclictest223473-21kworker/u8:3+events_unbound07:30:131
256064991816,1cyclictest448159-21cron10:15:000
256064991816,1cyclictest448159-21cron10:14:590
256064991815,2cyclictest516252-21kworker/u8:1+flush-259:012:30:000
256064991815,2cyclictest375515-21kworker/u8:2+events_unbound09:26:000
256064991814,3cyclictest0-21swapper/010:45:000
256064991814,1cyclictest60650irq/133-enp0s29f1:rx-007:35:160
256074991714,2cyclictest501260-21kworker/u8:4+flush-259:011:05:273
256074991714,2cyclictest317794-21kworker/u8:0+flush-259:008:40:193
256074991714,2cyclictest203158-21kworker/u8:1+flush-259:007:15:203
256074991714,2cyclictest0-21swapper/309:35:003
256074991713,3cyclictest558953-21kworker/u8:0+events_unbound12:00:193
256074991713,3cyclictest286946-21kworker/u8:1+events_unbound07:55:223
256074991713,3cyclictest2698871-21systemd-journal10:05:023
256069991715,2cyclictest317794-21kworker/u8:0+events_unbound08:16:382
256069991714,2cyclictest516253-21kworker/u8:2+events_unbound12:06:342
256069991714,2cyclictest516253-21kworker/u8:2+events_unbound11:26:432
256069991714,2cyclictest317794-21kworker/u8:0+events_unbound08:38:372
256069991714,2cyclictest276665-21kworker/u8:2+flush-259:007:40:202
256066991715,2cyclictest426975-21kworker/u8:2+events_unbound09:55:131
256066991715,2cyclictest364255-21kworker/u8:4+events_unbound08:55:221
256066991714,2cyclictest452541-21kworker/u8:2+events_unbound10:30:221
256066991714,2cyclictest203158-21kworker/u8:1+events_unbound07:10:221
256064991714,2cyclictest533223-21kworker/u8:4+events_unbound11:50:220
256064991714,2cyclictest516252-21kworker/u8:1+flush-259:012:35:160
256064991714,2cyclictest276665-21kworker/u8:2+flush-259:008:05:250
256064991714,2cyclictest223473-21kworker/u8:3+events_unbound07:30:160
256064991713,3cyclictest516252-21kworker/u8:1+events_unbound12:15:120
256064991711,1cyclictest458539-21turbostat10:25:000
256074991613,2cyclictest452541-21kworker/u8:2+events_unbound10:25:143
256074991613,2cyclictest307743-21kworker/u8:3+events_unbound08:00:233
256074991613,2cyclictest276665-21kworker/u8:2+flush-259:008:15:173
256074991612,3cyclictest480681-21apt-get10:45:133
256074991612,3cyclictest369733-21kworker/u8:0+events_unbound09:35:223
256069991614,1cyclictest364255-21kworker/u8:4+events_unbound09:20:012
256069991613,2cyclictest516253-21kworker/u8:2+flush-259:011:30:162
256069991613,2cyclictest516252-21kworker/u8:1+flush-259:012:12:462
256069991613,2cyclictest307743-21kworker/u8:3+events_unbound08:45:292
256069991612,3cyclictest533223-21kworker/u8:4+events_unbound11:50:422
256069991612,3cyclictest374979-21apt-key09:05:002
256066991613,2cyclictest452541-21kworker/u8:2+events_unbound10:20:181
256066991613,2cyclictest421309-21kworker/u8:3+flush-259:009:49:561
256066991613,2cyclictest203158-21kworker/u8:1+flush-259:007:15:261
256066991612,3cyclictest501271-21/usr/sbin/munin11:05:221
256064991614,1cyclictest60650irq/133-enp0s29f1:rx-008:02:280
256064991614,1cyclictest60650irq/133-enp0s29f1:rx-007:45:230
256064991613,2cyclictest60650irq/133-enp0s29f1:rx-009:05:020
256064991613,2cyclictest60650irq/133-enp0s29f1:rx-007:40:150
256064991613,2cyclictest533223-21kworker/u8:4+events_unbound11:40:250
256064991613,2cyclictest421309-21kworker/u8:3+flush-259:009:50:210
256064991613,2cyclictest421309-21kworker/u8:3+events_unbound09:49:280
256064991613,2cyclictest369733-21kworker/u8:0+events_unbound09:00:150
256064991613,2cyclictest317794-21kworker/u8:0+events_unbound08:34:410
256064991612,3cyclictest458558-21kworker/u8:1+events_unbound10:45:160
256064991611,4cyclictest558804-21apt-get12:00:000
256074991511,3cyclictest516252-21kworker/u8:1+flush-259:012:10:153
256069991513,2cyclictest432189-21kworker/u8:1+events_unbound10:10:212
256069991513,2cyclictest369733-21kworker/u8:0+events_unbound10:24:402
256069991513,1cyclictest0-21swapper/212:00:002
256069991512,2cyclictest589787-21kworker/u8:0+events_unbound12:32:442
256069991512,2cyclictest533223-21kworker/u8:4+flush-259:011:36:272
256069991512,2cyclictest516252-21kworker/u8:1+flush-259:012:35:162
256069991512,2cyclictest421309-21kworker/u8:3+events_unbound09:50:222
256069991512,2cyclictest395643-21kworker/u8:1+events_unbound09:45:172
256069991511,3cyclictest533223-21kworker/u8:4+events_unbound11:40:212
256069991511,3cyclictest517194-21gpgv11:20:012
256066991513,2cyclictest452541-21kworker/u8:2+events_unbound10:19:311
256066991512,2cyclictest276665-21kworker/u8:2+events_unbound08:20:351
256064991513,1cyclictest60650irq/133-enp0s29f1:rx-009:30:240
256064991513,1cyclictest60650irq/133-enp0s29f1:rx-007:20:170
256064991512,2cyclictest60650irq/133-enp0s29f1:rx-012:30:170
256064991512,2cyclictest522274-21munin-plugin-st11:25:010
256064991512,2cyclictest495427-21unixbench_singl10:55:270
256074991411,2cyclictest516252-21kworker/u8:1+flush-259:012:05:203
256074991411,2cyclictest473660-21kworker/u8:3+flush-259:011:10:203
256074991411,2cyclictest317794-21kworker/u8:0+flush-259:008:35:213
256074991411,1cyclictest375515-21kworker/u8:2-events_unbound09:44:593
256074991410,3cyclictest595497-21/usr/sbin/munin12:35:253
256074991410,3cyclictest432189-21kworker/u8:1+flush-259:010:10:163
256074991410,3cyclictest432189-21kworker/u8:1+flush-259:010:10:153
256069991411,2cyclictest458558-21kworker/u8:1+events_unbound10:26:002
256069991411,2cyclictest364255-21kworker/u8:4+events_unbound08:55:252
256069991411,2cyclictest333210-21kworker/u8:1+events_unbound08:50:432
256069991410,3cyclictest256029-21kworker/u8:0+events_unbound07:24:312
256066991412,2cyclictest375515-21kworker/u8:2+events_unbound09:10:091
256066991411,2cyclictest516253-21kworker/u8:2+events_unbound11:57:401
256066991410,3cyclictest533223-21kworker/u8:4+events_unbound11:36:521
256066991410,3cyclictest485818-21/usr/sbin/munin10:50:211
256064991413,1cyclictest60650irq/133-enp0s29f1:rx-007:25:070
256064991411,2cyclictest60650irq/133-enp0s29f1:rx-008:25:210
256064991411,2cyclictest375515-21kworker/u8:2+events_unbound09:20:200
256064991411,2cyclictest328738-21irqcore08:20:160
256064991411,2cyclictest317794-21kworker/u8:0+events_unbound08:50:210
256064991411,2cyclictest223473-21kworker/u8:3+events_unbound07:11:300
256064991411,2cyclictest0-21swapper/012:10:000
256074991310,2cyclictest533223-21kworker/u8:4+flush-259:011:55:253
256074991310,2cyclictest516253-21kworker/u8:2+flush-259:011:30:243
256074991310,2cyclictest419212-21kthreadcore09:45:193
25606999139,3cyclictest516253-21kworker/u8:2+events_unbound12:21:472
256069991312,1cyclictest286946-21kworker/u8:1+events_unbound07:57:582
25606699139,3cyclictest395643-21kworker/u8:1+events_unbound09:38:371
256064991310,2cyclictest60650irq/133-enp0s29f1:rx-009:41:540
256064991310,2cyclictest317794-21kworker/u8:0+events_unbound08:45:200
25607499129,2cyclictest458558-21kworker/u8:1+events_unbound10:40:233
25607499128,3cyclictest569549-21kworker/u8:3+events_unbound12:25:253
25607499128,3cyclictest375515-21kworker/u8:2+events_unbound09:20:193
256074991210,1cyclictest223473-21kworker/u8:3+flush-259:007:30:173
25606699129,2cyclictest516252-21kworker/u8:1+flush-259:012:35:261
25606699129,2cyclictest500651-21kworker/u8:0+flush-259:011:10:471
25607499118,2cyclictest369733-21kworker/u8:0+events_unbound09:55:243
25607499118,2cyclictest364892-21cpuspeed_turbos08:55:133
25607499118,2cyclictest307743-21kworker/u8:3+flush-259:008:10:213
25607499117,3cyclictest491ktimers/307:25:103
25607499106,3cyclictest375515-21kworker/u8:2+flush-259:009:10:173
25607499100,1cyclictest0-21swapper/310:20:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional