You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-20 - 06:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Mon Apr 20, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3047091995146,3cyclictest3248696-21gpgv22:25:012
3047086994844,3cyclictest3166159-21kworker/u8:2+flush-259:022:10:001
3047083994541,3cyclictest3331544-21kworker/u8:1+flush-259:000:15:170
3047083994541,3cyclictest3331544-21kworker/u8:1+flush-259:000:15:160
3047091994337,4cyclictest3367790-21latency_hist00:20:012
3047091994337,4cyclictest3367790-21latency_hist00:20:002
3047083994339,3cyclictest3062484-21kworker/u8:3+flush-259:020:40:170
3047083994135,4cyclictest3316299-21diskstats23:30:130
3047096994035,4cyclictest3099155-21/usr/sbin/munin20:00:163
3047083993831,5cyclictest3248487-21apt-get22:25:010
3047086993732,3cyclictest3161287-21apt-get21:00:141
304708399370,35cyclictest1-21systemd19:24:530
3047083993630,4cyclictest3098915-21latency_hist20:00:000
3047083993530,3cyclictest3197041-21kworker/u8:0+flush-259:022:30:180
3047091993429,4cyclictest3274696-21latency_hist22:50:022
3047086993429,4cyclictest3176487-21kworker/u8:3+flush-259:021:40:171
3047083993430,3cyclictest3166159-21kworker/u8:2+flush-259:021:45:180
3047083993429,3cyclictest252-21systemd-journal19:54:530
3047091993330,2cyclictest3166159-21kworker/u8:2+flush-259:023:05:172
3047086993327,4cyclictest3140442-21latency_hist20:40:001
3047083993329,3cyclictest3259456-21apt-get22:35:110
3047096993229,2cyclictest3197041-21kworker/u8:0+events_unbound23:10:183
3047096993226,4cyclictest252-21systemd-journal23:44:593
3047086993125,4cyclictest3124989-21kworker/u8:0+flush-259:020:44:591
3047086993125,4cyclictest3051282-21ntpq19:10:201
3047083993126,4cyclictest3331314-21latency_hist23:45:000
3047083993123,6cyclictest3294401-21ntpq23:05:220
3047096993027,2cyclictest3014210-21kworker/u8:2+flush-259:019:15:183
3047096993025,3cyclictest252-21systemd-journal00:25:013
3047083993026,3cyclictest3166159-21kworker/u8:2+flush-259:022:15:160
3047096992926,2cyclictest3166159-21kworker/u8:2+flush-259:022:50:163
3047091992926,2cyclictest3378237-21kworker/u8:3+events_unbound00:30:152
3047086992925,3cyclictest3285241-21apt-get23:00:121
3047086992924,3cyclictest252-21systemd-journal20:08:531
3047086992923,4cyclictest3191899-21latency_hist21:30:011
3047083992924,4cyclictest252-21systemd-journal22:05:010
304708399291,27cyclictest1-21systemd21:12:530
3047096992825,2cyclictest3062484-21kworker/u8:3+events_unbound19:55:153
3047096992825,1cyclictest3166159-21kworker/u8:2+events_unbound22:30:013
3047091992822,4cyclictest252-21systemd-journal20:11:282
3047096992720,5cyclictest3263506-21ntpq22:35:223
3047091992724,2cyclictest3062484-21kworker/u8:3+flush-259:019:35:162
3047086992723,3cyclictest3166159-21kworker/u8:2+events_unbound23:15:161
3047086992723,3cyclictest3062484-21kworker/u8:3+flush-259:020:35:001
3047096992623,2cyclictest3341648-21kworker/u8:2+flush-259:023:57:213
3047096992621,4cyclictest3259189-21latency_hist22:35:013
3047086992623,2cyclictest3151009-21apt-get20:50:121
3047083992623,2cyclictest3124989-21kworker/u8:0+flush-259:020:50:170
3047083992623,2cyclictest3014210-21kworker/u8:2+flush-259:019:15:160
3047083992622,3cyclictest3341648-21kworker/u8:2+events_unbound00:35:170
3047083992622,3cyclictest3341648-21kworker/u8:2+events_unbound00:35:160
3047083992622,3cyclictest3197041-21kworker/u8:0+events_unbound23:40:010
3047083992622,3cyclictest3176487-21kworker/u8:3+events_unbound21:40:150
3047096992522,2cyclictest3062484-21kworker/u8:3+flush-259:021:00:273
3047096992521,3cyclictest3362674-21kworker/u8:0+flush-259:000:25:203
3047096992521,3cyclictest3088681-21kworker/u8:1+flush-259:020:15:143
3047086992520,4cyclictest3321037-21kworker/u8:3+flush-259:023:55:241
3047086992520,4cyclictest3062484-21kworker/u8:3+flush-259:020:25:251
3047086992519,4cyclictest3119467-21latency_hist20:20:001
3047083992521,3cyclictest3062484-21kworker/u8:3+flush-259:020:55:260
3047083992520,3cyclictest252-21systemd-journal23:22:030
3047091992422,1cyclictest1-21systemd21:15:532
3047091992421,2cyclictest3062484-21kworker/u8:3+flush-259:020:30:182
3047091992421,2cyclictest3062484-21kworker/u8:3+events_unbound20:05:222
3047091992420,3cyclictest3362674-21kworker/u8:0+events_unbound00:25:192
3047091992419,4cyclictest3119655-21cp20:19:592
3047086992420,3cyclictest3331544-21kworker/u8:1+flush-259:023:49:591
3047086992420,3cyclictest3166159-21kworker/u8:2+flush-259:023:40:261
3047086992419,4cyclictest3057610-21apt-get19:20:121
3047083992419,4cyclictest3373350-21latency_hist00:25:000
3047096992320,2cyclictest3166159-21kworker/u8:2+events_unbound23:05:213
3047096992320,2cyclictest3166159-21kworker/u8:2+events_unbound21:20:223
3047096992319,3cyclictest3166159-21kworker/u8:2+events_unbound22:55:133
3047096992317,4cyclictest3228179-21apt-get22:05:113
3047083992319,3cyclictest3197041-21kworker/u8:0+events_unbound22:20:220
3047083992319,3cyclictest3166159-21kworker/u8:2+events_unbound22:55:160
3047083992319,3cyclictest252-21systemd-journal22:51:530
3047083992319,2cyclictest3014210-21kworker/u8:2+flush-259:019:10:150
3047096992220,2cyclictest3014210-21kworker/u8:2+events_unbound19:30:233
3047091992219,3cyclictest3197041-21kworker/u8:0+flush-259:021:50:202
3047091992218,3cyclictest3176487-21kworker/u8:3+flush-259:021:25:192
3047091992218,3cyclictest3124989-21kworker/u8:0+events_unbound20:51:542
3047091992215,5cyclictest252-21systemd-journal23:35:522
3047086992218,3cyclictest3166159-21kworker/u8:2+flush-259:021:10:001
3047083992219,2cyclictest3166159-21kworker/u8:2+events_unbound22:05:210
3047083992218,3cyclictest3166159-21kworker/u8:2+flush-259:023:15:210
3047083992217,4cyclictest3341648-21kworker/u8:2+events_unbound00:33:590
3047096992118,2cyclictest3197041-21kworker/u8:0+events_unbound23:19:553
3047096992118,2cyclictest3062484-21kworker/u8:3+flush-259:019:35:163
3047096992117,3cyclictest3166159-21kworker/u8:2+events_unbound21:36:433
3047096992117,3cyclictest3166159-21kworker/u8:2+events_unbound21:10:173
3047096992116,4cyclictest3315873-21latency_hist23:30:003
3047091992118,2cyclictest3197041-21kworker/u8:0+events_unbound22:31:522
3047091992118,2cyclictest3166159-21kworker/u8:2+events_unbound23:10:222
3047091992118,2cyclictest3031448-21kworker/u8:0+events_unbound19:25:012
3047091992117,3cyclictest3166159-21kworker/u8:2+events_unbound21:34:552
3047091992117,3cyclictest3124989-21kworker/u8:0+events_unbound21:13:552
3047091992117,3cyclictest3088681-21kworker/u8:1+events_unbound20:20:252
3047086992117,3cyclictest3197041-21kworker/u8:0+flush-259:022:25:251
3047086992116,4cyclictest3197041-21kworker/u8:0+flush-259:022:10:251
3047086992115,4cyclictest252-21systemd-journal22:30:541
3047086992113,6cyclictest3057346-21latency_hist19:20:001
3047086992113,5cyclictest252-21systemd-journal19:45:011
3047086992112,6cyclictest252-21systemd-journal20:05:011
3047083992118,2cyclictest3166159-21kworker/u8:2+events_unbound21:35:170
3047083992118,2cyclictest3014210-21kworker/u8:2+events_unbound19:25:240
3047083992117,3cyclictest3341648-21kworker/u8:2+flush-259:000:05:180
3047083992117,3cyclictest3197041-21kworker/u8:0+events_unbound21:50:260
3047083992117,3cyclictest3166159-21kworker/u8:2+flush-259:021:15:160
3047096992017,2cyclictest3388872-21idleruntime-cro00:39:593
3047096992017,2cyclictest3388872-21idleruntime-cro00:39:593
3047096992017,2cyclictest3197041-21kworker/u8:0+flush-259:021:50:213
3047096992016,3cyclictest3238453-21/usr/sbin/munin22:15:153
3047091992017,2cyclictest3321037-21kworker/u8:3+events_unbound23:55:142
3047091992016,3cyclictest3331544-21kworker/u8:1+events_unbound00:20:012
3047091992016,3cyclictest3197041-21kworker/u8:0+flush-259:023:00:222
3047091992016,3cyclictest3197041-21kworker/u8:0+flush-259:022:25:152
3047091992016,3cyclictest3197041-21kworker/u8:0+events_unbound22:36:522
3047091992016,3cyclictest3088681-21kworker/u8:1+events_unbound19:59:532
3047086992016,3cyclictest3031448-21kworker/u8:0+flush-259:019:25:261
3047086992015,4cyclictest3145823-21/usr/sbin/munin20:45:151
3047096991917,1cyclictest3142999-21kthreadcore20:40:183
3047096991916,3cyclictest3062484-21kworker/u8:3+events_unbound20:26:323
3047096991916,2cyclictest3124989-21kworker/u8:0+flush-259:020:48:573
3047096991916,2cyclictest3088681-21kworker/u8:1+events_unbound20:12:543
3047096991915,3cyclictest3321037-21kworker/u8:3+events_unbound23:50:233
3047096991915,3cyclictest3176487-21kworker/u8:3+events_unbound21:25:243
3047096991915,3cyclictest3062484-21kworker/u8:3+flush-259:020:30:143
3047091991916,3cyclictest3197041-21kworker/u8:0+flush-259:021:45:252
3047091991916,2cyclictest3341648-21kworker/u8:2+events_unbound00:35:182
3047091991916,2cyclictest3341648-21kworker/u8:2+events_unbound00:35:182
3047091991916,2cyclictest3331544-21kworker/u8:1+events_unbound00:05:542
3047091991916,2cyclictest3166159-21kworker/u8:2+flush-259:021:05:202
3047091991916,2cyclictest3166159-21kworker/u8:2+events_unbound23:34:532
3047091991916,2cyclictest3166159-21kworker/u8:2+events_unbound23:25:532
3047091991916,2cyclictest3014210-21kworker/u8:2+flush-259:019:40:192
3047091991915,3cyclictest3331544-21kworker/u8:1+events_unbound00:14:542
3047091991915,3cyclictest3331544-21kworker/u8:1+events_unbound00:05:002
3047091991915,3cyclictest3166159-21kworker/u8:2+flush-259:022:55:242
3047091991915,3cyclictest3166159-21kworker/u8:2+events_unbound23:15:202
3047091991915,3cyclictest3062484-21kworker/u8:3+events_unbound21:03:542
3047086991916,2cyclictest3124995-21cron20:25:001
3047086991912,5cyclictest3372465-21ntpq00:20:211
3047083991916,2cyclictest3321037-21kworker/u8:3+events_unbound23:55:210
3047083991916,2cyclictest3176487-21kworker/u8:3+events_unbound21:25:220
3047083991916,2cyclictest3062484-21kworker/u8:3+events_unbound20:34:540
3047096991814,3cyclictest3166159-21kworker/u8:2+flush-259:022:10:143
3047096991814,3cyclictest3166159-21kworker/u8:2+events_unbound23:39:563
3047091991816,1cyclictest3166159-21kworker/u8:2+events_unbound21:55:142
3047091991815,2cyclictest3166159-21kworker/u8:2+flush-259:023:20:202
3047091991814,3cyclictest3197041-21kworker/u8:0+events_unbound22:11:502
3047091991814,3cyclictest3124989-21kworker/u8:0+events_unbound20:38:542
3047086991814,3cyclictest3331544-21kworker/u8:1+flush-259:000:05:261
3047086991814,3cyclictest3166159-21kworker/u8:2+flush-259:021:10:261
3047086991813,4cyclictest3321037-21kworker/u8:3+flush-259:000:00:251
3047086991813,3cyclictest3062484-21kworker/u8:3+flush-259:019:55:011
3047086991811,5cyclictest252-21systemd-journal00:36:541
3047086991811,5cyclictest252-21systemd-journal00:36:531
3047083991814,3cyclictest3362674-21kworker/u8:0+events_unbound00:25:260
3047083991814,3cyclictest3088681-21kworker/u8:1+flush-259:020:00:210
3047096991714,2cyclictest3341648-21kworker/u8:2+flush-259:000:05:153
3047096991714,2cyclictest3331544-21kworker/u8:1+flush-259:000:00:583
3047096991714,2cyclictest3088681-21kworker/u8:1+flush-259:020:20:263
3047096991713,3cyclictest3083451-21latency_hist19:45:013
3047096991713,3cyclictest3062484-21kworker/u8:3+events_unbound20:05:143
3047096991713,2cyclictest0-21swapper/321:10:003
3047096991713,2cyclictest0-21swapper/300:20:003
3047096991713,2cyclictest0-21swapper/300:20:003
3047096991712,3cyclictest3166159-21kworker/u8:2+flush-259:023:20:163
3047096991711,5cyclictest3290112-21latency_hist23:05:023
3047096991711,4cyclictest3207578-21/usr/sbin/munin21:45:153
3047091991715,2cyclictest3197041-21kworker/u8:0+events_unbound22:41:542
3047091991714,3cyclictest3124989-21kworker/u8:0+flush-259:020:40:142
3047091991714,2cyclictest3166159-21kworker/u8:2+events_unbound21:35:192
3047091991714,2cyclictest3084127-21irqcore19:45:142
3047091991713,3cyclictest3176487-21kworker/u8:3+flush-259:021:40:252
3047091991713,3cyclictest3062484-21kworker/u8:3+flush-259:020:25:202
3047086991714,2cyclictest3181595-21idleruntime-cro21:20:001
3047086991713,3cyclictest3109457-21apt-get20:10:131
3047086991712,3cyclictest252-21systemd-journal21:35:531
3047083991715,1cyclictest3114328-21latency_hist20:14:590
3047096991613,3cyclictest3166159-21kworker/u8:2+events_unbound21:31:533
3047096991612,3cyclictest3197041-21kworker/u8:0+flush-259:022:45:133
3047096991612,3cyclictest3166159-21kworker/u8:2+flush-259:023:33:353
3047091991613,3cyclictest3197041-21kworker/u8:0+events_unbound22:52:532
3047091991613,2cyclictest3176487-21kworker/u8:3+events_unbound21:21:402
3047091991613,2cyclictest3014210-21kworker/u8:2+events_unbound19:50:172
3047091991612,3cyclictest3014210-21kworker/u8:2+events_unbound19:25:192
3047091991611,4cyclictest3166159-21kworker/u8:2+events_unbound22:04:552
304708699161,14cyclictest1-21systemd23:38:531
3047083991614,1cyclictest60350irq/132-enp0s29f1:rx-023:50:170
3047083991613,3cyclictest3341648-21kworker/u8:2+flush-259:000:10:230
3047083991613,2cyclictest60350irq/132-enp0s29f1:rx-020:19:280
3047083991613,2cyclictest60350irq/132-enp0s29f1:rx-019:30:200
3047083991613,2cyclictest3321037-21kworker/u8:3+events_unbound00:00:260
3047083991613,2cyclictest3166159-21kworker/u8:2+events_unbound21:35:010
3047083991612,3cyclictest3166159-21kworker/u8:2+events_unbound21:55:200
3047083991612,3cyclictest3135525-21/usr/sbin/munin20:35:160
3047083991612,3cyclictest3088681-21kworker/u8:1+events_unbound20:20:150
3047083991612,3cyclictest3062484-21kworker/u8:3+events_unbound20:25:240
3047083991612,3cyclictest3014210-21kworker/u8:2+events_unbound19:40:130
3047083991611,4cyclictest3150745-21latency_hist20:50:000
3047096991512,2cyclictest3358177-21irqcore00:10:173
3047096991512,2cyclictest3166159-21kworker/u8:2+events_unbound22:21:543
3047096991512,2cyclictest3124989-21kworker/u8:0+events_unbound20:35:203
3047096991511,3cyclictest3341648-21kworker/u8:2+events_unbound00:30:033
3047096991511,3cyclictest3014210-21kworker/u8:2+flush-259:019:25:033
3047096991510,3cyclictest3197041-21kworker/u8:0+events_unbound23:45:163
3047096991510,3cyclictest3124989-21kworker/u8:0+events_unbound20:55:223
3047091991512,2cyclictest3062484-21kworker/u8:3+events_unbound19:33:522
3047091991511,3cyclictest3124989-21kworker/u8:0+events_unbound20:55:142
3047086991512,2cyclictest3318428-21kthreadcore23:30:171
3047086991512,2cyclictest0-21swapper/123:24:591
3047083991513,2cyclictest60350irq/132-enp0s29f1:rx-021:02:460
3047083991513,1cyclictest60350irq/132-enp0s29f1:rx-023:10:230
3047083991513,1cyclictest60350irq/132-enp0s29f1:rx-022:40:260
3047083991512,2cyclictest60350irq/132-enp0s29f1:rx-020:05:240
3047083991512,2cyclictest60350irq/132-enp0s29f1:rx-019:35:200
3047083991512,2cyclictest3166159-21kworker/u8:2+flush-259:021:05:160
3047083991511,3cyclictest3197041-21kworker/u8:0+events_unbound23:45:200
304709699147,5cyclictest509-21rs:main3
3047096991410,3cyclictest3124989-21kworker/u8:0+events_unbound20:50:173
3047091991411,2cyclictest3088681-21kworker/u8:1+flush-259:020:00:182
3047091991410,3cyclictest3014210-21kworker/u8:2+events_unbound19:10:122
304708699149,4cyclictest3212694-21/usr/sbin/munin21:50:151
304708699149,3cyclictest3274938-21/usr/sbin/munin22:50:201
3047086991411,2cyclictest3341634-21http23:55:001
3047086991411,2cyclictest0-21swapper/121:25:001
3047086991410,3cyclictest3197041-21kworker/u8:0+flush-259:022:20:231
3047083991412,1cyclictest60350irq/132-enp0s29f1:rx-022:12:080
3047083991412,1cyclictest60350irq/132-enp0s29f1:rx-019:48:420
3047083991411,2cyclictest60350irq/132-enp0s29f1:rx-022:46:530
304709699139,3cyclictest3031448-21kworker/u8:0+flush-259:019:21:263
3047096991310,2cyclictest3176487-21kworker/u8:3+flush-259:021:40:283
3047096991310,2cyclictest3014210-21kworker/u8:2+flush-259:019:10:243
3047096991310,2cyclictest0-21swapper/321:20:003
304709199139,3cyclictest3197041-21kworker/u8:0+flush-259:022:15:182
304709199139,3cyclictest3166159-21kworker/u8:2+events_unbound23:40:212
3047091991310,2cyclictest3014210-21kworker/u8:2+events_unbound19:15:192
304708699139,3cyclictest3238700-21fschecks_time22:15:141
3047086991310,2cyclictest3299976-21unixbench_singl23:10:261
3047086991310,2cyclictest3073508-21cpuspeed_turbos19:35:121
3047086991310,2cyclictest252-21systemd-journal00:15:001
3047083991310,2cyclictest3166159-21kworker/u8:2+flush-259:023:25:000
3047083991310,2cyclictest3166159-21kworker/u8:2+events_unbound23:03:550
304709699129,2cyclictest3221789-21needreboot21:55:203
304709699129,2cyclictest3197041-21kworker/u8:0+events_unbound22:03:223
304709199129,3cyclictest3124989-21kworker/u8:0+events_unbound20:49:532
304708699129,1cyclictest0-21swapper/100:20:011
304708699129,1cyclictest0-21swapper/100:20:001
304708699127,4cyclictest3284699-21journal-offline22:56:531
304708699127,3cyclictest3067635-21apt-get19:30:021
304709199118,2cyclictest3331544-21kworker/u8:1+events_unbound23:49:552
304708699119,1cyclictest0-21swapper/122:00:001
304708699118,2cyclictest3378823-21cpuspeed_turbos00:30:151
304708699118,2cyclictest3166159-21kworker/u8:2+flush-259:023:05:211
304708699117,3cyclictest3197041-21kworker/u8:0+flush-259:022:45:261
304709199107,2cyclictest3166159-21kworker/u8:2+flush-259:022:05:242
304708699108,1cyclictest0-21swapper/122:05:001
304708699108,1cyclictest0-21swapper/119:45:231
304708699105,4cyclictest3062484-21kworker/u8:3+flush-259:019:55:241
30470869997,1cyclictest0-21swapper/121:45:241
30470869996,2cyclictest0-21swapper/122:40:131
30470869995,3cyclictest3156100-21/usr/sbin/munin20:55:161
30470869992,5cyclictest3192166-21apt-get21:30:131
30470869986,1cyclictest3311305-21if_enp0s29f123:25:171
30470869983,3cyclictest3197041-21kworker/u8:0+flush-259:022:35:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional