You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-03 - 15:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Fri Apr 03, 2026 00:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2937377994743,3cyclictest3259904-21kworker/u8:0+flush-259:000:40:002
2937368994438,4cyclictest3026172-21apt-get20:35:100
2937368994037,2cyclictest3057274-21kworker/u8:2+events_unbound21:10:020
2937372993733,3cyclictest3072960-21apt-get21:20:121
2937368993735,2cyclictest3015612-21kworker/u8:0+events_unbound21:00:250
2937368993731,4cyclictest3041933-21base6420:50:000
2937380993631,3cyclictest3093299-21gpgv21:40:003
2937380993631,3cyclictest3093299-21gpgv21:40:003
2937377993632,3cyclictest3135868-21kworker/u8:2+flush-259:022:50:152
2937372993625,10cyclictest3223194-21ntpq23:40:221
2937377993531,3cyclictest3187712-21kworker/u8:1+flush-259:023:25:222
2937377993531,3cyclictest3041351-21gpgv20:50:012
2937372993528,5cyclictest3104021-21latency_hist21:50:001
2937372993528,5cyclictest3104021-21latency_hist21:50:001
2937380993428,4cyclictest2947714-21kworker/u8:3+flush-259:020:00:153
2937368993430,3cyclictest3088318-21kworker/u8:4+events_unbound21:44:420
2937380993329,3cyclictest3193329-21gpgv23:15:003
2937377993228,3cyclictest3000007-21kworker/u8:1+flush-259:020:20:202
2937377993227,4cyclictest3239763-21apt-get00:00:122
2937377993226,4cyclictest3234618-21apt-get23:55:102
2937368993229,2cyclictest351319-21kworker/u8:1+events_unbound19:15:270
2937368993229,2cyclictest2974450-21kworker/u8:2+events_unbound19:57:200
2937377993128,2cyclictest3213776-21kworker/u8:0+flush-259:023:55:012
2937377993126,3cyclictest3146274-21kworker/u8:1+flush-259:022:40:212
2937377993124,5cyclictest3052113-21latency_hist21:00:002
2937368993128,2cyclictest3135868-21kworker/u8:2+events_unbound22:43:400
2937368993128,2cyclictest2947714-21kworker/u8:3+events_unbound20:42:410
2937377993026,3cyclictest3166619-21kworker/u8:0+events_unbound23:00:122
2937377993026,3cyclictest3067195-21kworker/u8:0+flush-259:021:35:002
2937377993025,3cyclictest3093425-21base6421:40:002
2937377993025,3cyclictest3093425-21base6421:40:002
2937372993027,2cyclictest351319-21kworker/u8:1+flush-259:019:10:161
2937368993028,2cyclictest3213776-21kworker/u8:0+events_unbound23:42:440
2937368993027,2cyclictest3234586-21kworker/u8:4+events_unbound00:12:430
2937368993027,2cyclictest3224120-21kworker/u8:2+events_unbound23:58:420
2937368993027,2cyclictest3057275-21kworker/u8:3+events_unbound21:19:420
2937368993024,4cyclictest252-21systemd-journal23:24:590
2937377992923,4cyclictest252-21systemd-journal23:46:412
2937368992926,2cyclictest3259904-21kworker/u8:0+events_unbound00:34:300
2937368992926,2cyclictest3234586-21kworker/u8:4+events_unbound00:03:430
2937368992926,2cyclictest2947714-21kworker/u8:3+events_unbound19:27:190
2937380992823,3cyclictest3146274-21kworker/u8:1+flush-259:022:45:013
2937377992824,3cyclictest3249843-21kworker/u8:1+events_unbound00:30:282
2937377992824,3cyclictest3249843-21kworker/u8:1+events_unbound00:25:242
2937377992823,3cyclictest3213776-21kworker/u8:0+events_unbound23:40:192
2937368992826,2cyclictest3015612-21kworker/u8:0+events_unbound20:49:400
2937377992724,2cyclictest3057274-21kworker/u8:2+flush-259:021:15:252
2937377992724,2cyclictest2963727-21kworker/u8:1+events_unbound19:35:132
2937377992723,3cyclictest3057274-21kworker/u8:2+events_unbound21:05:272
2937377992723,3cyclictest3015612-21kworker/u8:0+flush-259:021:05:002
2937377992723,3cyclictest2947714-21kworker/u8:3+flush-259:020:35:152
2937377992722,3cyclictest3067195-21kworker/u8:0+events_unbound22:10:182
2937377992721,4cyclictest2984706-21latency_hist19:55:012
2937372992723,3cyclictest3224120-21kworker/u8:2+flush-259:023:55:121
2937368992724,2cyclictest3146274-21kworker/u8:1+events_unbound23:01:410
2937368992724,2cyclictest3146274-21kworker/u8:1+events_unbound22:35:280
2937380992622,3cyclictest3000007-21kworker/u8:1+flush-259:020:20:163
2937380992620,4cyclictest509-21rs:main3
2937377992622,3cyclictest3187712-21kworker/u8:1+flush-259:023:35:192
2937377992620,4cyclictest3077816-21latency_hist21:25:002
2937372992620,5cyclictest3181673-21ntpq23:00:211
2937368992623,2cyclictest3146274-21kworker/u8:1+events_unbound22:52:400
2937368992623,2cyclictest3135868-21kworker/u8:2+events_unbound23:06:170
2937368992623,2cyclictest3057274-21kworker/u8:2+events_unbound21:07:400
2937380992519,4cyclictest3031284-21/usr/sbin/munin20:40:233
2937380992516,6cyclictest252-21systemd-journal22:20:003
2937377992521,3cyclictest2947714-21kworker/u8:3+flush-259:020:00:212
2937368992521,3cyclictest252-21systemd-journal20:32:190
2937368992520,4cyclictest3083237-21apt-get21:30:110
2937368992519,4cyclictest2973558-21ntpq19:40:210
2937380992419,4cyclictest3146274-21kworker/u8:1+flush-259:022:50:113
2937380992419,3cyclictest3214068-21apt-get23:35:123
2937377992421,2cyclictest2947714-21kworker/u8:3+events_unbound19:55:202
2937377992421,2cyclictest2947714-21kworker/u8:3+events_unbound19:25:212
2937377992419,4cyclictest3015612-21kworker/u8:0+events_unbound20:50:042
293737299244,10cyclictest3068779-21sed21:15:181
2937372992418,4cyclictest3041331-21latency_hist20:50:011
293737299241,16cyclictest3129918-21taskset22:12:011
2937368992421,2cyclictest3099066-21kworker/u8:3+events_unbound22:13:420
2937368992421,2cyclictest2947714-21kworker/u8:3+events_unbound19:39:570
2937368992420,3cyclictest351319-21kworker/u8:1+events_unbound19:11:210
2937368992419,4cyclictest3082085-21ntpq21:25:220
2937380992318,4cyclictest3161741-21/usr/sbin/munin22:45:153
2937377992320,2cyclictest3135868-21kworker/u8:2+flush-259:023:05:182
2937377992318,3cyclictest3249843-21kworker/u8:1+flush-259:000:20:282
293737299234,8cyclictest3257799-21grep00:15:181
2937372992319,3cyclictest2947714-21kworker/u8:3+flush-259:020:25:161
2937368992320,2cyclictest3135868-21kworker/u8:2+events_unbound22:21:210
2937368992320,2cyclictest3000007-21kworker/u8:1+events_unbound20:24:040
2937368992318,4cyclictest3146274-21kworker/u8:1+flush-259:023:00:000
2937380992218,3cyclictest3213776-21kworker/u8:0+flush-259:023:50:163
2937380992218,3cyclictest3067195-21kworker/u8:0+flush-259:021:45:213
2937380992218,3cyclictest3067195-21kworker/u8:0+flush-259:021:45:213
2937380992216,4cyclictest3224120-21kworker/u8:2+flush-259:023:55:153
2937380992216,4cyclictest2974450-21kworker/u8:2+flush-259:020:05:173
2937377992219,2cyclictest3067195-21kworker/u8:0+flush-259:022:00:212
2937377992219,2cyclictest3067195-21kworker/u8:0+events_unbound22:15:202
2937377992218,3cyclictest3187712-21kworker/u8:1+flush-259:023:20:252
2937377992218,3cyclictest3077853-21kworker/u8:1+flush-259:021:25:222
2937377992218,3cyclictest2947714-21kworker/u8:3+events_unbound19:30:222
2937377992218,3cyclictest2947714-21kworker/u8:3+events_unbound19:30:222
2937377992217,4cyclictest2937346-21kworker/u8:0+flush-259:019:20:102
2937372992219,2cyclictest509-21rs:main1
2937372992217,3cyclictest509-21rs:main1
2937372992216,4cyclictest2968823-21latency_hist19:40:011
293737299220,11cyclictest3165583-21needreboot22:45:201
2937368992220,2cyclictest3234586-21kworker/u8:4+events_unbound00:07:580
2937368992219,2cyclictest3259904-21kworker/u8:0+events_unbound00:24:500
2937368992219,2cyclictest3213776-21kworker/u8:0+events_unbound23:51:040
2937368992219,2cyclictest3213776-21kworker/u8:0+events_unbound23:45:080
2937368992219,2cyclictest3187712-21kworker/u8:1+events_unbound23:18:170
2937368992219,2cyclictest3182343-21kworker/u8:3+events_unbound23:13:220
2937368992219,2cyclictest3067195-21kworker/u8:0+events_unbound22:09:280
2937368992219,2cyclictest3000007-21kworker/u8:1+events_unbound20:59:350
2937368992219,2cyclictest2974450-21kworker/u8:2+flush-259:019:50:170
2937380992114,5cyclictest3040412-21ntpq20:45:193
2937380992113,6cyclictest3150308-21ntpq22:30:223
2937380992111,5cyclictest252-21systemd-journal00:10:013
2937377992118,2cyclictest3234586-21kworker/u8:4+flush-259:000:07:462
2937372992117,3cyclictest3260364-21apt-get00:20:111
2937372992116,4cyclictest3088084-21latency_hist21:35:001
293737299210,12cyclictest3031690-21cat20:40:151
293737299210,11cyclictest3120053-21apt-config22:05:021
293737299210,10cyclictest2952698-21modprobe19:21:401
2937368992118,2cyclictest3275538-21kworker/u8:3+events_unbound00:35:590
2937368992118,2cyclictest3213776-21kworker/u8:0+events_unbound23:37:010
2937368992118,2cyclictest3146274-21kworker/u8:1+events_unbound22:32:350
2937368992118,2cyclictest3067195-21kworker/u8:0+events_unbound21:24:420
2937368992118,2cyclictest3000007-21kworker/u8:1+events_unbound20:17:530
2937368992118,2cyclictest2947714-21kworker/u8:3+events_unbound19:22:500
2937368992117,3cyclictest3077853-21kworker/u8:1+events_unbound21:51:240
2937380992015,3cyclictest3000007-21kworker/u8:1+flush-259:021:00:013
2937380992013,5cyclictest252-21systemd-journal21:52:363
2937377992018,2cyclictest3067195-21kworker/u8:0+flush-259:021:40:222
2937377992017,2cyclictest3146274-21kworker/u8:1+events_unbound22:35:252
2937377992016,3cyclictest3187712-21kworker/u8:1+events_unbound23:17:482
2937377992016,3cyclictest3166619-21kworker/u8:0+events_unbound22:55:112
2937377992016,3cyclictest3099066-21kworker/u8:3+events_unbound22:25:152
2937377992016,3cyclictest3099066-21kworker/u8:3+events_unbound22:20:132
2937377992016,3cyclictest3041363-21kworker/u8:4+events_unbound21:10:202
2937372992013,5cyclictest252-21systemd-journal00:35:001
293737299200,15cyclictest3109123-21seq21:54:481
293737299200,14cyclictest3139600-21ntpq22:20:241
293737299200,10cyclictest3233434-21ld23:50:191
2937368992017,2cyclictest3234586-21kworker/u8:4+events_unbound00:17:060
2937368992017,2cyclictest3135868-21kworker/u8:2+events_unbound22:26:260
2937368992017,2cyclictest3067195-21kworker/u8:0+events_unbound22:00:330
2937368992017,2cyclictest3000007-21kworker/u8:1+events_unbound20:13:210
2937368992017,2cyclictest2947714-21kworker/u8:3+events_unbound20:09:270
2937368992016,3cyclictest3187712-21kworker/u8:1+events_unbound23:33:100
2937368992016,3cyclictest3135868-21kworker/u8:2+events_unbound22:48:230
2937368992016,3cyclictest252-21systemd-journal22:16:400
2937368992015,4cyclictest3113448-21ntpq21:55:200
2937380991915,3cyclictest507-21in:imuxsock23:10:013
2937380991914,3cyclictest3249843-21kworker/u8:1+flush-259:000:10:213
2937380991914,3cyclictest3057274-21kworker/u8:2+flush-259:021:05:253
2937380991913,4cyclictest3067195-21kworker/u8:0+flush-259:022:20:263
2937377991916,2cyclictest3187712-21kworker/u8:1+flush-259:023:30:102
2937377991916,2cyclictest3146274-21kworker/u8:1+flush-259:022:35:002
2937377991915,3cyclictest3259904-21kworker/u8:0+events_unbound00:18:442
2937377991912,5cyclictest3005709-21diskstats20:15:132
293737299191,8cyclictest3009586-21ntpq20:15:211
293737299190,9cyclictest3186779-21ntpq23:05:201
293737299190,9cyclictest3161308-21sshd22:42:441
293737299190,9cyclictest2947808-21seq19:19:571
293737299190,10cyclictest3176909-21taskset22:59:491
2937368991917,2cyclictest3249843-21kworker/u8:1+events_unbound00:26:500
2937368991917,2cyclictest3182343-21kworker/u8:3+events_unbound23:25:480
2937368991916,2cyclictest3088318-21kworker/u8:4+events_unbound21:36:340
2937368991916,2cyclictest3088318-21kworker/u8:4+events_unbound21:36:340
2937368991916,2cyclictest3000007-21kworker/u8:1+events_unbound20:27:090
2937368991916,2cyclictest2974450-21kworker/u8:2+events_unbound20:00:470
2937380991814,3cyclictest2947714-21kworker/u8:3+flush-259:019:55:163
2937380991813,3cyclictest3120142-21/usr/sbin/munin22:05:233
2937377991814,3cyclictest3099066-21kworker/u8:3+events_unbound21:55:202
2937377991814,3cyclictest3067195-21kworker/u8:0+events_unbound21:45:232
2937377991814,3cyclictest3067195-21kworker/u8:0+events_unbound21:45:222
293737299189,7cyclictest3214166-21cpuspeed_turbos23:35:131
2937372991814,3cyclictest252-21systemd-journal19:57:151
293737299180,9cyclictest3171538-21seq22:52:171
293737299180,9cyclictest3051989-21taskset20:58:391
293737299180,9cyclictest3015270-21chrt20:21:201
293737299180,8cyclictest3145272-21seq22:25:571
293737299180,13cyclictest3114238-21chrt21:59:131
293737299180,12cyclictest3151053-21chrt22:33:191
293737299180,10cyclictest3280503-21chrt00:37:411
293737299180,10cyclictest3020986-21chrt20:30:101
293737299180,10cyclictest2984448-21seq19:52:021
2937368991816,2cyclictest2947714-21kworker/u8:3+events_unbound19:47:420
2937368991815,2cyclictest3067195-21kworker/u8:0+events_unbound21:45:530
2937368991815,2cyclictest3067195-21kworker/u8:0+events_unbound21:45:530
2937368991815,2cyclictest2937346-21kworker/u8:0+events_unbound19:31:250
2937368991815,2cyclictest2937346-21kworker/u8:0+events_unbound19:31:250
2937380991714,2cyclictest0-21swapper/321:45:003
2937380991713,3cyclictest3057274-21kworker/u8:2+flush-259:021:30:213
2937377991714,2cyclictest3099066-21kworker/u8:3+events_unbound22:05:162
2937377991714,2cyclictest3015612-21kworker/u8:0+events_unbound20:30:232
2937377991714,2cyclictest2947714-21kworker/u8:3+flush-259:020:40:212
2937377991713,3cyclictest2974450-21kworker/u8:2+flush-259:020:08:442
293737299178,1cyclictest0-21swapper/122:20:001
2937372991712,3cyclictest252-21systemd-journal23:16:411
293737299170,9cyclictest3187927-21/usr/sbin/munin23:10:141
293737299170,9cyclictest2969483-21runrttasks19:40:081
293737299170,8cyclictest3092918-21taskset21:35:511
293737299170,8cyclictest3092918-21taskset21:35:511
293737299170,7cyclictest2979558-21munin-run19:50:001
293737299170,11cyclictest3119453-21seq22:04:531
2937380991614,1cyclictest3082966-21idleruntime-cro21:30:003
2937380991613,2cyclictest2969933-21irqcore19:40:163
2937380991610,4cyclictest3125465-21diskstats22:10:143
2937380991610,4cyclictest252-21systemd-journal21:10:403
2937377991614,1cyclictest3000007-21kworker/u8:1+events_unbound20:10:192
2937377991613,2cyclictest3015612-21kworker/u8:0+events_unbound20:30:012
2937377991612,3cyclictest3146274-21kworker/u8:1+flush-259:022:45:202
2937377991612,3cyclictest2937346-21kworker/u8:0+events_unbound19:44:412
2937377991610,4cyclictest3109155-21latency_hist21:55:002
293737299161,13cyclictest3043026-21grep20:50:151
293737299161,13cyclictest2937096-21latency_hist19:10:011
293737299160,9cyclictest3244351-21taskset00:01:261
293737299160,8cyclictest3269559-21ntpq00:25:231
293737299160,8cyclictest3254594-21taskset00:10:421
293737299160,8cyclictest3213743-21seq23:34:301
293737299160,8cyclictest3093935-21/usr/sbin/munin21:40:161
293737299160,7cyclictest3245225-21hddtemp_smartct00:05:151
293737299160,7cyclictest3082886-21chrt21:29:081
293738099158,5cyclictest3005287-21latency_hist20:15:013
2937380991512,2cyclictest3025907-21sh20:35:003
293738099151,12cyclictest490-21dbus-daemon21:25:003
2937380991510,4cyclictest2947714-21kworker/u8:3+flush-259:020:15:113
2937380991510,4cyclictest2942951-21apt-get19:15:113
2937377991511,3cyclictest3182343-21kworker/u8:3+events_unbound23:12:422
2937377991511,3cyclictest2937346-21kworker/u8:0+events_unbound19:14:402
293737299154,9cyclictest3001508-21cat20:10:171
293737299151,7cyclictest3228965-21taskset23:47:101
293737299150,7cyclictest832-21runrttasks20:08:131
293737299150,7cyclictest3208414-21chrt23:27:081
293737299150,7cyclictest3203291-21chrt23:22:401
293737299150,7cyclictest2963312-21seq19:30:391
293737299150,7cyclictest2963312-21seq19:30:381
293737299150,6cyclictest3031055-21munin-run20:40:011
293737299150,13cyclictest3057600-21expr21:05:141
293737299150,12cyclictest3156468-21latency_hist22:40:021
293737299150,12cyclictest3057259-21latency_hist21:05:001
293738099149,3cyclictest3259904-21kworker/u8:0+flush-259:000:25:163
2937380991411,2cyclictest3209042-21cpuspeed_turbos23:30:113
2937380991411,2cyclictest3118581-21ntp_kernel_pll_22:00:213
2937380991410,3cyclictest3155872-21unixbench_multi22:35:273
2937380991410,3cyclictest2971-21cron00:35:013
2937377991411,2cyclictest351319-21kworker/u8:1+events_unbound19:15:192
2937377991410,3cyclictest3234586-21kworker/u8:4+events_unbound00:12:522
293737299140,12cyclictest3063096-21sort21:10:171
2937368991411,2cyclictest0-21swapper/019:09:590
293738099139,3cyclictest2937346-21kworker/u8:0+flush-259:019:26:043
293738099139,2cyclictest0-21swapper/300:00:223
293738099131,10cyclictest3280711-21awk00:40:003
2937380991310,2cyclictest0-21swapper/319:39:593
293738099129,2cyclictest3225365-21tr23:45:183
293738099128,3cyclictest3187712-21kworker/u8:1+flush-259:023:20:283
293738099128,3cyclictest2947714-21kworker/u8:3+flush-259:019:51:133
293738099126,4cyclictest3234586-21kworker/u8:4+flush-259:000:15:243
293738099111,8cyclictest3208662-21awk23:29:593
293738099107,1cyclictest0-21swapper/322:55:253
293738099101,8cyclictest2979550-21turbostat.cron19:50:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional