You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 00:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Fri Apr 24, 2026 12:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2827197995148,2cyclictest2910371-21kworker/u8:4+events_unbound09:05:242
2827197994744,2cyclictest2889784-21kworker/u8:0+events_unbound09:20:132
2827197994543,2cyclictest3034829-21kworker/u8:0+events_unbound10:35:262
2827197994441,2cyclictest3066106-21kworker/u8:1+events_unbound11:05:142
2827197994441,2cyclictest2993459-21kworker/u8:1+events_unbound09:50:222
2827197994441,2cyclictest2843129-21kworker/u8:0+events_unbound07:35:132
2827190994439,3cyclictest3024273-21kworker/u8:3+events_unbound10:30:151
2827183994429,13cyclictest2977607-21/usr/sbin/munin09:35:230
2827183994429,13cyclictest2977607-21/usr/sbin/munin09:35:230
2827197994239,2cyclictest3066106-21kworker/u8:1+events_unbound11:20:152
2827197994239,2cyclictest3003971-21kworker/u8:2+events_unbound10:15:112
2827197994239,2cyclictest2905209-21kworker/u8:1+events_unbound08:40:232
2827197994138,2cyclictest3003971-21kworker/u8:2+events_unbound10:30:122
2827197994138,2cyclictest2843129-21kworker/u8:0+events_unbound07:50:142
2827190994137,3cyclictest3024273-21kworker/u8:3+flush-259:010:25:191
2827197994037,2cyclictest2889784-21kworker/u8:0+events_unbound08:30:132
2827190994033,5cyclictest3117276-21ntpq11:45:221
2827190993935,3cyclictest2794044-21kworker/u8:1+flush-259:007:15:001
2827197993835,2cyclictest3024273-21kworker/u8:3+events_unbound10:45:132
2827197993832,2cyclictest2889784-21kworker/u8:0+events_unbound09:15:232
2827190993834,3cyclictest2806274-21kworker/u8:3+flush-259:007:30:201
2827199993732,3cyclictest3091912-21latency_hist11:25:013
2827197993735,1cyclictest3123994-21kworker/u8:2+events_unbound12:10:252
2827197993733,3cyclictest3003971-21kworker/u8:2+events_unbound10:50:232
2827190993731,2cyclictest2889784-21kworker/u8:0+flush-259:008:25:221
2827197993633,2cyclictest3123994-21kworker/u8:2+events_unbound12:20:222
2827197993633,2cyclictest2977599-21kworker/u8:2+events_unbound09:45:142
2827197993633,2cyclictest2977599-21kworker/u8:2+events_unbound09:45:132
2827190993632,3cyclictest3024273-21kworker/u8:3+flush-259:011:39:591
2827190993632,3cyclictest2982562-21gpgv09:40:011
2827190993632,3cyclictest2982562-21gpgv09:40:001
2827190993632,3cyclictest2879266-21kworker/u8:1+flush-259:008:00:231
2827183993631,3cyclictest2910609-21apt-get08:30:100
2827197993532,2cyclictest2889784-21kworker/u8:0+events_unbound09:00:232
2827197993532,2cyclictest2879644-21kworker/u8:3+events_unbound08:05:212
2827197993531,3cyclictest3155290-21kworker/u8:0+events_unbound12:30:182
2827197993531,3cyclictest2889784-21kworker/u8:0+events_unbound08:54:252
2827190993531,3cyclictest2843129-21kworker/u8:0+flush-259:007:30:001
2827197993431,2cyclictest3024273-21kworker/u8:3+events_unbound11:34:262
2827190993430,3cyclictest2889784-21kworker/u8:0+flush-259:008:55:221
2827190993430,3cyclictest2827155-21kworker/u8:2+flush-259:007:40:251
2827183993428,5cyclictest3159644-21ntpq12:25:210
2827197993330,2cyclictest3123994-21kworker/u8:2+events_unbound12:00:122
2827197993330,2cyclictest2806274-21kworker/u8:3+events_unbound07:20:002
2827190993330,2cyclictest3097453-21kworker/u8:0+events_unbound12:05:151
2827190993329,3cyclictest3107756-21kworker/u8:2+events_unbound11:40:121
2827190993329,3cyclictest2869275-21kworker/u8:4+flush-259:007:55:191
2827183993328,3cyclictest3050906-21apt-get10:45:110
2827197993229,2cyclictest3139204-21kworker/u8:1+events_unbound12:16:252
2827197993229,2cyclictest3066106-21kworker/u8:1+events_unbound11:17:252
2827197993229,2cyclictest3024273-21kworker/u8:3+events_unbound10:25:142
2827197993228,3cyclictest2879644-21kworker/u8:3+events_unbound08:03:252
2827197993227,4cyclictest2889784-21kworker/u8:0+events_unbound08:12:252
2827197993226,4cyclictest2900081-21latency_hist08:20:022
2827190993228,3cyclictest3139204-21kworker/u8:1+flush-259:012:20:211
2827199993127,3cyclictest3003971-21kworker/u8:2+flush-259:010:35:003
2827197993129,2cyclictest3097453-21kworker/u8:0+events_unbound11:35:142
2827197993128,2cyclictest3024273-21kworker/u8:3+events_unbound11:51:262
2827197993128,2cyclictest3024273-21kworker/u8:3+events_unbound11:25:242
2827190993128,2cyclictest2827155-21kworker/u8:2+flush-259:007:45:231
2827190993127,3cyclictest2910371-21kworker/u8:4+flush-259:009:55:191
2827190993127,3cyclictest2889784-21kworker/u8:0+flush-259:008:30:211
2827183993125,4cyclictest3050642-21latency_hist10:45:000
2827190993025,3cyclictest3024273-21kworker/u8:3+events_unbound11:16:271
2827190993024,4cyclictest2869307-21apt-get07:50:101
2827197992926,2cyclictest2889784-21kworker/u8:0+flush-259:008:37:282
2827197992925,3cyclictest2843129-21kworker/u8:0+events_unbound07:57:242
2827190992925,3cyclictest3040656-21apt-get10:35:131
2827190992925,3cyclictest2905209-21kworker/u8:1+flush-259:008:40:171
2827190992924,4cyclictest3149719-21apt-get12:20:001
2827190992924,4cyclictest3066106-21kworker/u8:1+flush-259:011:05:251
2827183992917,9cyclictest3024273-21kworker/u8:3+flush-259:011:50:010
2827199992823,4cyclictest3118142-21apt-get11:50:003
2827197992825,2cyclictest3066106-21kworker/u8:1+events_unbound11:10:172
2827197992825,2cyclictest3024273-21kworker/u8:3+events_unbound11:40:192
2827190992825,2cyclictest3123994-21kworker/u8:2+events_unbound12:35:141
2827190992825,2cyclictest3003971-21kworker/u8:2+events_unbound10:50:131
2827190992824,3cyclictest2889784-21kworker/u8:0+flush-259:009:25:201
2827190992822,4cyclictest2951629-21latency_hist09:10:011
2827199992721,4cyclictest3134177-21latency_hist12:05:013
2827199992720,5cyclictest3107805-21latency_hist11:40:013
2827197992724,2cyclictest3139204-21kworker/u8:1+events_unbound12:28:252
2827197992724,2cyclictest2910371-21kworker/u8:4+events_unbound09:40:222
2827197992724,2cyclictest2879644-21kworker/u8:3+events_unbound08:22:232
2827197992723,3cyclictest2853322-21gpgv07:35:012
2827190992723,3cyclictest3024273-21kworker/u8:3+flush-259:011:30:211
2827199992622,3cyclictest2889784-21kworker/u8:0+events_unbound09:50:153
2827197992623,2cyclictest3003971-21kworker/u8:2+events_unbound10:04:502
2827197992622,3cyclictest3117764-21kworker/u8:4+events_unbound12:05:252
2827197992622,3cyclictest3024273-21kworker/u8:3+events_unbound11:47:262
2827197992622,3cyclictest3003971-21kworker/u8:2+flush-259:010:05:212
2827197992622,3cyclictest2889784-21kworker/u8:0+flush-259:009:10:202
2827197992621,4cyclictest2826916-21latency_hist07:10:002
2827197992621,4cyclictest2826916-21latency_hist07:10:002
2827190992623,2cyclictest2889784-21kworker/u8:0+flush-259:009:50:201
2827190992622,3cyclictest3139204-21kworker/u8:1+flush-259:012:25:241
2827190992622,3cyclictest3066106-21kworker/u8:1+events_unbound11:20:131
2827190992622,3cyclictest3024273-21kworker/u8:3+flush-259:011:30:001
2827190992622,3cyclictest3003971-21kworker/u8:2+events_unbound10:00:101
2827190992622,3cyclictest2905209-21kworker/u8:1+flush-259:008:35:111
2827190992620,2cyclictest3024273-21kworker/u8:3+events_unbound10:45:201
2827183992619,5cyclictest2878559-21ntpq07:55:220
2827183992615,10cyclictest2900504-21diskstats08:20:120
2827183992611,13cyclictest2889767-21turbostat08:10:000
2827199992521,3cyclictest3092469-21gpgv11:25:023
2827199992519,4cyclictest2832563-21latency_hist07:15:003
2827197992522,2cyclictest3024273-21kworker/u8:3+events_unbound10:40:252
2827197992522,2cyclictest2910371-21kworker/u8:4+events_unbound09:27:252
2827197992520,3cyclictest252-21systemd-journal07:47:242
2827183992521,3cyclictest3024273-21kworker/u8:3+flush-259:011:10:240
2827183992519,4cyclictest2863910-21latency_hist07:45:010
2827190992421,2cyclictest2879644-21kworker/u8:3+events_unbound08:20:201
2827190992421,2cyclictest2806274-21kworker/u8:3+events_unbound07:20:221
2827183992418,5cyclictest2894077-21ntpq08:10:220
2827183992418,4cyclictest2936402-21/usr/sbin/munin08:55:120
2827183992418,4cyclictest252-21systemd-journal09:59:590
2827183992413,10cyclictest3014233-21turbostat10:10:010
2827199992318,3cyclictest2967073-21latency_hist09:25:003
2827190992320,2cyclictest3003971-21kworker/u8:2+flush-259:010:10:141
2827190992320,2cyclictest2972230-21kworker/u8:1+flush-259:009:30:231
2827190992319,3cyclictest2946511-21kworker/u8:1+events_unbound09:15:131
2827190992319,3cyclictest2910371-21kworker/u8:4+flush-259:009:40:251
2827190992319,3cyclictest2910371-21kworker/u8:4+events_unbound08:50:231
2827190992317,5cyclictest3024273-21kworker/u8:3+flush-259:010:44:551
282718399239,12cyclictest3029204-21unixbench_multi10:20:230
282718399236,4cyclictest513-21gdbus09:20:000
2827183992319,3cyclictest2879677-21apt-get08:00:110
2827199992219,2cyclictest3003971-21kworker/u8:2+events_unbound10:48:173
2827197992220,2cyclictest2910371-21kworker/u8:4+events_unbound09:56:252
2827197992219,2cyclictest3066106-21kworker/u8:1+events_unbound11:04:592
2827197992219,2cyclictest2910371-21kworker/u8:4+events_unbound10:10:012
2827197992219,2cyclictest2843129-21kworker/u8:0+events_unbound07:43:232
2827197992218,3cyclictest2806274-21kworker/u8:3+events_unbound07:21:252
2827190992218,3cyclictest2843129-21kworker/u8:0+events_unbound07:36:521
2827190992217,3cyclictest2889784-21kworker/u8:0+flush-259:009:10:301
2827190992216,4cyclictest3066106-21kworker/u8:1+flush-259:011:10:151
2827183992219,2cyclictest3097453-21kworker/u8:0+flush-259:011:55:160
2827183992218,3cyclictest2889784-21kworker/u8:0+flush-259:009:25:240
2827183992215,6cyclictest2847109-21ntpq07:25:190
2827190992117,3cyclictest3097453-21kworker/u8:0+events_unbound11:50:221
2827190992117,3cyclictest3003971-21kworker/u8:2+events_unbound10:20:221
2827190992117,3cyclictest3003971-21kworker/u8:2+events_unbound10:08:251
2827190992115,3cyclictest3003971-21kworker/u8:2+events_unbound10:15:191
282718399217,1cyclictest0-21swapper/012:10:000
282718399215,5cyclictest2993228-21latency_hist09:50:000
282718399215,5cyclictest2993228-21latency_hist09:50:000
2827183992116,3cyclictest2853203-21apt-get07:35:000
2827183992115,5cyclictest2919827-21ntpq08:35:210
2827183992114,5cyclictest252-21systemd-journal11:30:010
2827183992113,5cyclictest252-21systemd-journal12:34:590
2827183992112,9cyclictest60350irq/132-enp0s29f1:rx-007:37:500
282718399211,17cyclictest0-21swapper/010:35:120
2827199992018,1cyclictest0-21swapper/310:05:003
2827199992015,4cyclictest2847892-40mandb07:28:273
2827199992014,5cyclictest2941324-21latency_hist09:00:003
2827197992017,2cyclictest2920692-21kworker/u8:2+events_unbound08:49:242
2827197992017,2cyclictest2889784-21kworker/u8:0+events_unbound09:35:032
2827197992017,2cyclictest2889784-21kworker/u8:0+events_unbound09:35:022
2827197992015,4cyclictest2848039-21latency_hist07:30:002
2827197992010,1cyclictest191rcu_preempt11:59:592
2827190992017,2cyclictest2910371-21kworker/u8:4+events_unbound09:46:241
2827190992017,2cyclictest2910371-21kworker/u8:4+events_unbound09:46:231
2827190992014,4cyclictest3066080-21latency_hist11:00:001
282718399208,11cyclictest0-21swapper/007:20:130
282718399206,2cyclictest3112955-21sh11:45:000
2827183992016,2cyclictest0-21swapper/009:45:000
282718399201,4cyclictest3066074-21idleruntime-cro11:00:000
2827183992011,8cyclictest60350irq/132-enp0s29f1:rx-009:04:590
2827183992011,7cyclictest3098763-21kthreadcore11:30:170
282718399200,6cyclictest3066349-21taskset11:00:120
2827197991916,2cyclictest2794044-21kworker/u8:1+events_unbound07:11:222
2827190991915,3cyclictest3123994-21kworker/u8:2+flush-259:011:55:211
2827190991915,3cyclictest3066106-21kworker/u8:1+events_unbound11:04:581
282718399196,3cyclictest17250irq/128-nvme0q111:05:240
282718399196,1cyclictest2947708-21sed09:05:170
282718399191,5cyclictest2874253-21apt07:55:000
282718399191,5cyclictest252-21systemd-journal10:30:000
282718399191,4cyclictest3134179-21apt12:05:000
2827183991912,5cyclictest3023600-21ntpq10:15:190
2827183991911,6cyclictest2976542-21ntpq09:30:210
2827199991815,2cyclictest2920659-21idleruntime-cro08:40:003
2827199991815,1cyclictest491ktimers/307:30:203
2827199991814,3cyclictest3039959-21apt-get10:35:033
2827199991814,3cyclictest2869275-21kworker/u8:4+flush-259:007:55:243
2827199991813,3cyclictest2837702-21latency_hist07:20:003
2827197991815,2cyclictest3024273-21kworker/u8:3+events_unbound10:56:242
2827190991813,4cyclictest3123994-21kworker/u8:2+events_unbound12:10:171
2827190991813,3cyclictest2879644-21kworker/u8:3+events_unbound08:10:191
282718399189,8cyclictest60350irq/132-enp0s29f1:rx-011:15:170
282718399187,2cyclictest2864579-21cut07:45:150
282718399180,3cyclictest2951865-21/usr/sbin/munin09:10:130
282719999173,8cyclictest3013330-21ntpq10:05:193
282719999172,4cyclictest2873346-21ld07:50:193
2827197991714,2cyclictest3155290-21kworker/u8:0+events_unbound12:36:242
2827197991714,2cyclictest2972230-21kworker/u8:1+events_unbound09:34:242
2827197991713,3cyclictest2910371-21kworker/u8:4+flush-259:008:58:492
2827190991714,2cyclictest3155290-21kworker/u8:0+events_unbound12:32:241
282718399171,6cyclictest3039096-21ntpq10:30:200
2827183991714,2cyclictest3166322-21irqcore12:35:160
2827183991713,3cyclictest3123994-21kworker/u8:2+flush-259:012:20:240
2827183991711,4cyclictest3060933-21idleruntime-cro10:55:000
282719999162,2cyclictest2930103-21ntpq08:45:223
2827199991615,1cyclictest2889784-21kworker/u8:0+events_unbound09:19:373
2827199991612,3cyclictest3124029-21apt-get11:55:113
2827199991612,3cyclictest3066106-21kworker/u8:1+flush-259:011:44:593
2827199991612,3cyclictest2972230-21kworker/u8:1+flush-259:009:30:223
2827197991613,2cyclictest2905209-21kworker/u8:1+events_unbound08:26:192
2827190991613,2cyclictest2920692-21kworker/u8:2+events_unbound08:45:161
2827190991613,2cyclictest2879644-21kworker/u8:3+events_unbound08:15:141
2827190991613,2cyclictest2794044-21kworker/u8:1+events_unbound07:15:191
2827190991612,3cyclictest2889784-21kworker/u8:0+events_unbound09:00:261
2827183991613,2cyclictest60350irq/132-enp0s29f1:rx-011:52:400
2827183991613,2cyclictest60350irq/132-enp0s29f1:rx-011:24:490
2827183991613,2cyclictest3149910-21gpgconf12:20:000
2827183991611,4cyclictest60350irq/132-enp0s29f1:rx-012:13:570
2827183991611,4cyclictest60350irq/132-enp0s29f1:rx-009:22:550
2827183991611,4cyclictest60350irq/132-enp0s29f1:rx-007:15:130
2827183991611,4cyclictest2993459-21kworker/u8:1+flush-259:009:50:250
2827183991611,3cyclictest60350irq/132-enp0s29f1:rx-011:35:110
282719999156,5cyclictest3071270-21munin-run11:05:003
282719999152,2cyclictest50-21ksoftirqd/309:45:143
282719999152,2cyclictest50-21ksoftirqd/309:45:133
282719999152,2cyclictest481rcuc/312:05:123
2827199991514,1cyclictest2889784-21kworker/u8:0+events_unbound08:18:063
2827199991513,2cyclictest2879266-21kworker/u8:1+events_unbound08:03:513
2827199991512,1cyclictest191rcu_preempt10:50:263
2827199991511,3cyclictest2879266-21kworker/u8:1+flush-259:008:15:003
2827199991510,3cyclictest252-21systemd-journal09:15:003
282719999150,4cyclictest2868131-21ntpq07:45:203
282719799155,2cyclictest191rcu_preempt10:20:172
2827190991512,2cyclictest2879266-21kworker/u8:1+events_unbound08:07:261
2827183991513,1cyclictest60350irq/132-enp0s29f1:rx-008:45:490
2827183991513,1cyclictest60350irq/132-enp0s29f1:rx-008:15:210
2827183991513,1cyclictest60350irq/132-enp0s29f1:rx-007:13:080
2827183991512,2cyclictest60350irq/132-enp0s29f1:rx-008:25:240
2827183991512,2cyclictest2936163-21idleruntime-cro08:55:010
282719999149,1cyclictest191rcu_preempt12:35:143
282719999148,3cyclictest0-21swapper/311:31:133
282719999143,2cyclictest50-21ksoftirqd/310:40:183
282719999142,3cyclictest50-21ksoftirqd/312:20:123
282719999142,2cyclictest50-21ksoftirqd/309:55:143
282719999142,2cyclictest491ktimers/310:10:203
282719999141,6cyclictest50-21ksoftirqd/310:55:133
282719999141,3cyclictest3033511-21cut10:25:173
2827199991412,1cyclictest481rcuc/308:23:233
2827199991411,2cyclictest2977894-21fschecks_time09:35:143
2827199991411,2cyclictest2977894-21fschecks_time09:35:133
2827199991411,1cyclictest191rcu_preempt12:27:473
2827199991411,1cyclictest191rcu_preempt08:27:533
2827190991411,2cyclictest3123994-21kworker/u8:2+events_unbound12:00:181
2827190991410,3cyclictest2889784-21kworker/u8:0+events_unbound09:20:021
2827183991412,1cyclictest60350irq/132-enp0s29f1:rx-008:40:230
282719999138,3cyclictest0-21swapper/307:42:413
282719999137,3cyclictest0-21swapper/312:34:553
282719999136,4cyclictest0-21swapper/310:15:473
282719999132,3cyclictest50-21ksoftirqd/309:40:103
282719999132,2cyclictest3143597-21ntpq12:10:203
282719999131,4cyclictest50-21ksoftirqd/307:35:213
282719999131,2cyclictest50-21ksoftirqd/311:15:203
2827199991311,1cyclictest491ktimers/309:05:133
2827199991310,2cyclictest3081549-21cron11:15:013
2827199991310,1cyclictest191rcu_preempt12:15:443
2827199991310,1cyclictest0-21swapper/311:10:013
282719999130,7cyclictest3024768-21/usr/sbin/munin10:20:143
282719999130,7cyclictest2925031-21ntpq08:40:213
282719999130,4cyclictest2888856-21ld08:05:213
2827183991310,2cyclictest60350irq/132-enp0s29f1:rx-010:03:430
2827183991310,2cyclictest3019367-21idleruntime-cro10:15:000
2827183991310,2cyclictest0-21swapper/007:10:000
282719999123,1cyclictest191rcu_preempt09:29:133
282719999113,5cyclictest0-21swapper/309:00:203
282719999111,3cyclictest50-21ksoftirqd/307:20:153
282719999110,3cyclictest50-21ksoftirqd/308:50:183
282719999110,2cyclictest3122899-21ntpq11:50:223
282719099100,3cyclictest2826916-21latency_hist07:10:001
282719099100,3cyclictest2826916-21latency_hist07:10:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional