You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 19:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Tue Jul 21, 2026 12:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2917715995549,4cyclictest3021125-21kworker/u8:2+events_unbound12:20:212
2917706995333,17cyclictest2937011-21auditctl08:05:000
2917718994934,10cyclictest2919626-21fschecks_count07:15:023
2917706994938,7cyclictest2933579-21cron07:55:000
2917718994838,7cyclictest3015890-21grep11:50:263
291770699481,42cyclictest2945773-21apt-get08:30:150
291770699480,30cyclictest0-21swapper/008:25:150
2917718994736,7cyclictest3005431-21grep11:20:253
2917718994736,7cyclictest3005431-21grep11:20:253
2917718994713,8cyclictest2948944-21sh08:40:013
2917710994741,4cyclictest2973195-21kworker/u8:0+events_unbound09:49:581
291770699470,38cyclictest0-21swapper/012:05:120
291770699470,32cyclictest0-21swapper/007:10:260
291770699460,38cyclictest0-21swapper/008:10:120
291771899453,10cyclictest2992859-21/usr/sbin/munin10:45:213
291770699450,36cyclictest0-21swapper/012:30:140
2917718994436,5cyclictest2990298-21latency_hist10:40:003
2917718994414,25cyclictest0-21swapper/308:00:183
291770699441,38cyclictest0-21swapper/009:15:190
291771899433,7cyclictest3000527-21cron11:10:013
2917718994322,16cyclictest0-21swapper/308:30:283
2917718994319,21cyclictest145-21jbd2/mmcblk1p2-811:55:193
2917718994313,4cyclictest0-21swapper/312:30:273
2917710994337,4cyclictest2933478-21kworker/u8:1+events_unbound07:54:261
291770699430,37cyclictest0-21swapper/010:30:120
291771899429,5cyclictest2963500-21/usr/sbin/munin09:20:173
291771899429,5cyclictest0-21swapper/309:39:563
291771899422,10cyclictest2994975-21tail10:50:253
291771899421,10cyclictest2958124-21apt-get09:05:063
291771899421,10cyclictest2927993-21unixbench_multi07:35:263
291771899421,10cyclictest2927993-21unixbench_multi07:35:263
2917710994236,4cyclictest2921617-21kworker/u8:1+events_unbound07:44:251
2917710994236,4cyclictest2921617-21kworker/u8:1+events_unbound07:44:251
2917710994233,5cyclictest3031879-21munin-plugin-st12:40:001
291770699422,35cyclictest2985434-21apt-get10:25:140
291770699421,38cyclictest3022423-21/usr/sbin/munin12:10:220
291770699421,38cyclictest3001539-21ntp_offset11:10:210
291770699421,38cyclictest2977585-21/usr/sbin/munin10:00:240
291770699421,36cyclictest0-21swapper/010:10:250
291770699420,38cyclictest0-21swapper/011:05:120
291770699420,37cyclictest0-21swapper/011:50:120
291771899412,7cyclictest3010994-21sendmail11:40:003
291771899412,10cyclictest2976190-21tail09:55:293
291771899412,10cyclictest2976190-21tail09:55:283
291771899411,7cyclictest2960053-21cut09:10:163
2917718994114,23cyclictest0-21swapper/307:25:253
291771899411,11cyclictest2998388-21unixbench_multi11:00:263
2917710994135,4cyclictest2973195-21kworker/u8:0+events_unbound10:00:001
2917710994135,4cyclictest2973195-21kworker/u8:0+events_unbound10:00:001
2917710994133,6cyclictest231ktimers/112:25:121
291770699412,34cyclictest3012256-21grep11:40:270
291770699411,37cyclictest3031044-21/usr/sbin/munin12:35:230
291770699411,37cyclictest3028529-21latency_hist12:30:010
291770699411,36cyclictest2982316-21/usr/sbin/munin10:15:170
291770699410,32cyclictest0-21swapper/011:00:130
291771899402,6cyclictest3003703-21grep11:15:283
291771899401,9cyclictest3026757-21cron12:25:003
291771899401,8cyclictest2988129-21unixbench_multi10:30:253
2917718994014,23cyclictest2930567-21fschecks_count07:45:173
291771899401,11cyclictest2939061-21diskstats08:10:153
291771899401,11cyclictest2938704-21cron08:10:003
291771899401,11cyclictest2934877-21unixbench_singl07:55:273
291771899401,11cyclictest2934877-21unixbench_singl07:55:273
291771899401,11cyclictest2925480-21fschecks_time07:30:173
2917718994010,4cyclictest0-21swapper/312:00:253
2917715994031,6cyclictest2968127-21cron09:35:002
2917715994031,5cyclictest208-21systemd-logind08:24:292
2917706994029,7cyclictest2955556-21grep08:55:280
291770699401,36cyclictest3018943-21/usr/sbin/munin12:00:240
291770699401,36cyclictest3004499-21/usr/sbin/munin11:20:140
291770699401,36cyclictest3004499-21/usr/sbin/munin11:20:140
291770699401,36cyclictest2984071-21/usr/sbin/munin10:20:170
291770699401,36cyclictest2979247-21/usr/sbin/munin10:05:250
291770699401,36cyclictest2975536-21/usr/sbin/munin09:55:210
291770699401,36cyclictest2975536-21/usr/sbin/munin09:55:210
291770699401,36cyclictest2968815-21/usr/sbin/munin09:35:180
291770699401,36cyclictest2952083-21/usr/sbin/munin08:45:270
291770699401,36cyclictest2950216-21/usr/sbin/munin08:40:270
291770699401,36cyclictest2942651-21/usr/sbin/munin08:20:190
291770699401,36cyclictest2941797-21sshd08:16:190
291770699401,36cyclictest2924622-21/usr/sbin/munin07:25:280
291770699401,34cyclictest2970218-21/usr/sbin/munin09:40:150
291770699401,34cyclictest2937485-21/usr/sbin/munin08:05:190
291770699400,32cyclictest0-21swapper/009:45:140
291770699400,32cyclictest0-21swapper/007:35:130
291770699400,32cyclictest0-21swapper/007:35:130
291771899393,8cyclictest204-21dbus-daemon09:49:573
291771899392,6cyclictest164-21systemd-journal07:24:193
291771899392,13cyclictest2974969-21cut09:55:003
2917718993916,20cyclictest645-21runrttasks09:00:213
2917718993916,20cyclictest645-21runrttasks09:00:213
2917718993912,3cyclictest2966662-21/usr/sbin/munin09:30:263
291771899391,11cyclictest2983476-21cron10:20:023
291771899391,10cyclictest2952100-21grep08:45:273
2917715993932,5cyclictest3003877-21kworker/u8:1+events_unbound11:20:102
2917715993932,5cyclictest3003877-21kworker/u8:1+events_unbound11:20:102
2917710993933,4cyclictest2955590-21kworker/u8:2+events_unbound09:29:561
2917710993933,4cyclictest2933478-21kworker/u8:1+events_unbound08:04:291
291770699392,34cyclictest2958748-21/usr/sbin/munin09:05:250
291770699392,33cyclictest2934390-21ntp_states07:55:230
291770699392,33cyclictest2934390-21ntp_states07:55:230
291770699391,35cyclictest3006724-21/usr/sbin/munin11:25:250
291770699391,35cyclictest2988922-21/usr/sbin/munin10:35:130
291770699391,35cyclictest2965729-21/usr/sbin/munin09:25:240
291770699391,35cyclictest2960181-21/usr/sbin/munin09:10:180
291770699391,34cyclictest2974317-21sed09:50:250
291770699391,33cyclictest645-21runrttasks07:40:140
291770699391,33cyclictest645-21runrttasks07:40:130
291770699391,33cyclictest2952801-21munin-node08:50:120
291770699390,33cyclictest0-21swapper/007:20:140
291771899389,4cyclictest0-21swapper/312:05:253
291771899383,6cyclictest2997123-21sh11:00:003
291771899382,8cyclictest3031412-21grep12:35:263
291771899382,10cyclictest2952939-21fschecks_count08:50:163
291771899381,9cyclictest2966025-21grep09:25:273
291771899381,9cyclictest2962551-21grep09:15:283
291771899381,9cyclictest2930366-21latency_hist07:45:013
291771899381,9cyclictest2930366-21latency_hist07:45:013
291771899381,8cyclictest2984766-21head10:20:243
291771899381,8cyclictest2941698-21grep08:15:263
291771899381,6cyclictest3027183-21cut12:25:163
2917718993812,2cyclictest0-21swapper/307:51:083
2917718993811,3cyclictest2954502-21/usr/sbin/munin08:55:233
291771899381,11cyclictest3002034-21sed11:10:263
291771899381,10cyclictest2980063-21sshd10:09:583
291771899381,10cyclictest2950612-21sshd08:44:353
2917715993810,6cyclictest131rcu_preempt12:35:122
2917710993832,4cyclictest2929855-21kworker/u8:2+events_unbound07:59:281
2917710993832,4cyclictest2929855-21kworker/u8:2+events_unbound07:59:271
2917710993830,4cyclictest0-21swapper/109:40:151
2917710993819,6cyclictest131rcu_preempt10:50:231
2917706993826,6cyclictest0-21swapper/010:55:230
291770699381,34cyclictest3025815-21/usr/sbin/munin12:20:190
291770699381,34cyclictest3013592-21meminfo11:45:190
291770699381,34cyclictest2995031-21/usr/sbin/munin10:50:260
291770699381,34cyclictest2993727-21basename10:50:000
291770699381,33cyclictest2990551-21apt10:40:110
291770699381,32cyclictest3018152-21latency_hist12:00:010
291770699381,32cyclictest2947654-21/usr/sbin/munin08:35:170
291770699380,35cyclictest2966665-21apt-get09:30:120
291771899372,6cyclictest3007149-21grep11:25:283
291771899372,6cyclictest2986893-21cron10:30:013
291771899371,8cyclictest3007802-21sshd11:30:103
291771899371,6cyclictest2943837-21debian-sa108:25:003
2917718993714,5cyclictest391ktimers/310:43:393
291771899371,11cyclictest3024032-21meminfo12:15:213
291771899371,11cyclictest2977292-21wc10:00:193
2917715993730,4cyclictest0-21swapper/210:40:002
2917710993727,6cyclictest3010569-21unixbench_singl11:35:271
2917710993722,5cyclictest3009515-21kworker/u8:0+events_unbound11:52:311
291770699372,32cyclictest2930869-21meminfo07:45:210
291770699371,33cyclictest3009508-21apt-get11:35:130
291770699371,33cyclictest2925636-21irqstats07:30:190
291770699371,33cyclictest2919879-21munin-node07:15:020
291770699371,33cyclictest2919879-21munin-node07:15:020
291770699371,32cyclictest3007809-21/usr/sbin/munin11:30:110
291770699371,32cyclictest2956341-21diskstats09:00:150
291770699371,32cyclictest2956341-21diskstats09:00:150
291770699370,34cyclictest3024024-21/usr/sbin/munin12:15:200
291771899362,6cyclictest2970615-21tr09:40:203
291771899361,9cyclictest3014177-21grep11:45:263
291771899361,10cyclictest2921172-21grep07:15:273
291771899361,10cyclictest2921172-21grep07:15:273
2917715993629,4cyclictest0-21swapper/207:40:172
2917715993629,4cyclictest0-21swapper/207:40:172
2917715993627,6cyclictest2978374-21cron10:05:002
2917710993629,4cyclictest2948976-21idleruntime-cro08:40:011
2917710993625,7cyclictest2977962-21grep10:00:261
291770699361,32cyclictest3002793-21df11:15:150
2917718993510,3cyclictest3011610-21latency11:40:203
2917715993528,3cyclictest0-21swapper/212:10:002
2917715993524,8cyclictest311ktimers/210:40:132
2917710993517,10cyclictest131rcu_preempt10:58:411
291770699351,31cyclictest2963383-21diskstats09:20:170
291771899342,6cyclictest3022894-21unixbench_multi12:10:263
2917715993427,4cyclictest2974908-21kworker/u8:2+events_unbound10:14:592
2917715993427,4cyclictest0-21swapper/207:25:282
2917710993428,4cyclictest2919250-21kworker/u8:1+events_unbound07:14:151
2917710993425,5cyclictest2949401-21grep08:40:171
2917710993424,5cyclictest0-21swapper/111:40:171
2917710993422,4cyclictest131rcu_preempt08:45:251
2917710993416,11cyclictest131rcu_preempt07:30:151
291771899337,5cyclictest2944826-21cat08:25:263
291771899333,6cyclictest2981372-21grep10:10:263
2917715993325,4cyclictest0-21swapper/208:05:282
2917715993319,5cyclictest2973195-21kworker/u8:0+events_unbound09:50:202
291771099337,9cyclictest24-21ksoftirqd/109:40:001
2917710993326,5cyclictest2955590-21kworker/u8:2+events_unbound08:58:411
2917710993326,4cyclictest0-21swapper/108:30:281
2917710993325,4cyclictest0-21swapper/109:30:161
2917710993321,9cyclictest3007563-21cron11:30:001
2917710993320,4cyclictest131rcu_preempt10:33:021
2917710993318,5cyclictest131rcu_preempt11:45:251
2917710993317,5cyclictest131rcu_preempt07:50:001
2917710993315,11cyclictest131rcu_preempt12:20:011
2917710993315,11cyclictest131rcu_preempt10:30:011
2917710993314,4cyclictest131rcu_preempt09:50:221
291771599323,3cyclictest131rcu_preempt10:45:102
2917715993217,5cyclictest3022991-21kworker/u8:3+events_unbound12:15:522
291771599320,4cyclictest131rcu_preempt08:15:122
2917710993225,4cyclictest0-21swapper/110:15:261
2917710993224,4cyclictest0-21swapper/112:25:001
2917710993224,4cyclictest0-21swapper/112:00:171
2917710993222,6cyclictest3020411-21modprobe12:05:181
2917710993215,9cyclictest131rcu_preempt09:12:541
2917710993215,3cyclictest131rcu_preempt11:55:091
2917710993210,11cyclictest131rcu_preempt11:30:171
2917715993125,4cyclictest2983322-21kworker/u8:0+events_unbound10:56:072
2917715993123,4cyclictest0-21swapper/209:35:232
2917715993119,8cyclictest204-21dbus-daemon11:55:172
291771599311,5cyclictest131rcu_preempt08:00:142
291771099315,9cyclictest24-21ksoftirqd/111:20:001
291771099315,15cyclictest24-21ksoftirqd/107:35:251
291771099315,15cyclictest24-21ksoftirqd/107:35:251
2917710993124,4cyclictest2921648-21cron07:20:021
2917710993124,4cyclictest2921648-21cron07:20:011
2917710993117,4cyclictest131rcu_preempt08:50:231
2917710993116,5cyclictest131rcu_preempt11:20:171
2917710993116,5cyclictest131rcu_preempt11:20:161
2917710993116,5cyclictest131rcu_preempt11:14:001
2917710993115,5cyclictest131rcu_preempt12:30:231
2917710993115,5cyclictest131rcu_preempt10:44:191
2917710993114,5cyclictest131rcu_preempt08:30:011
2917710993112,5cyclictest131rcu_preempt09:20:181
2917710993111,12cyclictest24-21ksoftirqd/110:10:231
2917710993110,4cyclictest0-21swapper/109:15:221
291771599302,5cyclictest131rcu_preempt09:20:132
2917715993023,4cyclictest0-21swapper/209:49:572
2917715993021,7cyclictest3012965-21latency_hist11:45:012
2917715993020,6cyclictest0-21swapper/208:35:282
2917710993022,4cyclictest0-21swapper/107:20:171
2917710993017,4cyclictest131rcu_preempt11:00:471
2917710993016,3cyclictest0-21swapper/110:49:041
2917710993014,5cyclictest131rcu_preempt08:05:221
2917710993013,9cyclictest131rcu_preempt10:35:191
2917710993013,5cyclictest131rcu_preempt11:10:001
2917710993011,9cyclictest131rcu_preempt09:05:521
2917715992922,3cyclictest0-21swapper/209:00:262
2917715992922,3cyclictest0-21swapper/209:00:262
2917715992918,5cyclictest3009515-21kworker/u8:0+events_unbound11:53:422
291771599291,2cyclictest0-21swapper/207:10:132
2917715992912,7cyclictest32-21ksoftirqd/208:40:242
2917710992922,5cyclictest2933478-21kworker/u8:1+events_unbound08:19:301
2917710992915,4cyclictest131rcu_preempt08:10:251
2917710992913,5cyclictest131rcu_preempt09:00:201
2917710992913,5cyclictest131rcu_preempt09:00:201
2917710992913,5cyclictest131rcu_preempt08:20:201
2917710992910,10cyclictest0-21swapper/112:10:261
291771599282,4cyclictest0-21swapper/207:35:142
291771599282,4cyclictest0-21swapper/207:35:142
2917715992821,4cyclictest0-21swapper/211:45:262
2917715992820,6cyclictest2955590-21kworker/u8:2+events_unbound08:56:422
2917715992818,7cyclictest2985200-21auditctl10:25:012
2917710992811,10cyclictest131rcu_preempt10:20:201
2917715992719,4cyclictest0-21swapper/211:15:002
2917715992715,5cyclictest32-21ksoftirqd/211:25:252
291771099274,9cyclictest24-21ksoftirqd/107:25:191
2917715992617,6cyclictest0-21swapper/207:50:232
2917715992616,6cyclictest0-21swapper/209:15:002
2917715992615,3cyclictest32-21ksoftirqd/207:45:132
2917715992612,6cyclictest32-21ksoftirqd/210:50:232
291771599259,3cyclictest131rcu_preempt11:15:132
2917715992519,4cyclictest2955590-21kworker/u8:2+events_unbound09:05:132
2917715992519,4cyclictest2940321-21kworker/u8:2+events_unbound08:34:342
2917715992519,3cyclictest629-21snmpd09:15:202
2917715992519,3cyclictest0-21swapper/212:30:002
2917715992518,5cyclictest2915544-21kworker/u8:0+events_unbound07:19:262
2917715992518,5cyclictest2915544-21kworker/u8:0+events_unbound07:19:262
291771599251,7cyclictest0-21swapper/211:35:132
2917715992512,4cyclictest3028815-21df_inode12:30:172
2917715992512,3cyclictest0-21swapper/211:00:142
2917715992511,6cyclictest32-21ksoftirqd/210:15:172
291771599244,4cyclictest131rcu_preempt11:30:262
2917715992418,4cyclictest2933478-21kworker/u8:1+events_unbound07:57:532
2917715992418,4cyclictest2933478-21kworker/u8:1+events_unbound07:57:532
2917715992418,3cyclictest0-21swapper/212:00:172
2917715992417,5cyclictest2921617-21kworker/u8:1+events_unbound07:24:582
2917715992417,4cyclictest0-21swapper/212:10:262
2917715992417,4cyclictest0-21swapper/209:55:282
2917715992417,4cyclictest0-21swapper/209:55:282
2917715992416,6cyclictest2952574-21munin-plugin-st08:50:002
2917715992414,8cyclictest32-21ksoftirqd/210:25:242
2917715992414,6cyclictest0-21swapper/211:05:272
2917715992413,3cyclictest32-21ksoftirqd/207:30:272
2917715992412,10cyclictest32-21ksoftirqd/210:30:182
2917715992317,4cyclictest2955590-21kworker/u8:2+events_unbound09:41:522
2917715992313,4cyclictest131rcu_preempt08:25:262
2917715992313,3cyclictest32-21ksoftirqd/210:05:272
2917715992313,3cyclictest32-21ksoftirqd/208:10:182
291771599227,5cyclictest2955590-21kworker/u8:2+events_unbound09:25:462
2917715992211,8cyclictest32-21ksoftirqd/208:50:182
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional