You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 17:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Mon Apr 13, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3460404994839,5cyclictest3491857-21sendmail08:40:010
3460412994740,4cyclictest3522982-21ls10:10:012
3460407994717,27cyclictest3479157-21sort08:00:251
3460407994536,5cyclictest3491844-21sendmail08:40:001
3460404994533,10cyclictest121ktimers/008:40:110
3460415994432,8cyclictest391ktimers/311:15:123
3460412994430,10cyclictest311ktimers/209:30:132
3460407994435,6cyclictest231ktimers/108:45:101
3460404994432,8cyclictest3476272-21cron07:55:000
3460415994330,8cyclictest3470703-21grep07:35:253
346040799432,8cyclictest3496503-21tail08:50:271
3460407994313,6cyclictest3545601-21gpgv11:15:011
3460404994333,6cyclictest3535141-21munin-plugin-st10:45:000
346040799426,11cyclictest3572806-21grep12:30:251
3460407994233,6cyclictest231ktimers/110:55:131
3460407994230,8cyclictest0-21swapper/111:20:221
3460407994230,8cyclictest0-21swapper/111:20:221
3460407994213,4cyclictest3486915-21fschecks_count08:25:161
346040799421,10cyclictest3533801-21cut10:40:171
346040799421,10cyclictest3493548-21cron08:45:001
3460415994134,4cyclictest3464330-21cron07:20:013
3460412994133,4cyclictest0-21swapper/209:00:002
346040799419,5cyclictest3547319-21seq11:18:061
346040799418,9cyclictest3482685-21grep08:10:251
346040799418,9cyclictest3482685-21grep08:10:251
3460407994115,3cyclictest3523205-21/usr/sbin/munin10:10:211
3460407994115,23cyclictest3488680-21fschecks_time08:30:191
3460407994115,22cyclictest3507437-21idleruntime-cro09:25:011
346040799411,12cyclictest3484339-21grep08:15:251
3460407994110,4cyclictest3464336-21sh07:20:011
3460415994032,4cyclictest0-21swapper/308:45:013
3460412994026,11cyclictest311ktimers/211:25:132
346040799401,9cyclictest3516146-21cron09:50:011
346040799401,7cyclictest3501670-21grep09:05:251
3460407994012,4cyclictest0-21swapper/109:50:231
3460407994011,3cyclictest3573503-21/usr/sbin/munin12:35:191
3460404994032,5cyclictest3542178-21idleruntime-cro11:05:000
346040499401,5cyclictest131rcu_preempt08:25:130
3460415993932,4cyclictest3495266-21idleruntime-cro08:50:003
346040799392,7cyclictest3538270-21grep10:50:271
3460407993927,8cyclictest3503593-21grep09:10:241
3460407993915,21cyclictest0-21swapper/110:30:001
3460407993915,21cyclictest0-21swapper/110:30:001
3460407993911,5cyclictest0-21swapper/112:10:131
3460407993911,3cyclictest0-21swapper/112:05:131
346040799391,11cyclictest3519099-21grep09:55:251
346040799391,10cyclictest3525924-21tail10:15:271
3460404993932,4cyclictest3505709-21munin-plugin-st09:20:000
346040799382,13cyclictest3505691-21cron09:20:011
346040799381,9cyclictest3552183-21head11:30:271
3460407993814,3cyclictest3566681-21/usr/sbin/munin12:15:171
3460404993830,5cyclictest3564757-21munin-plugin-st12:10:000
3460415993728,5cyclictest3556032-21munin-plugin-st11:45:003
3460415993727,6cyclictest3515743-21grep09:45:253
3460412993717,5cyclictest0-21swapper/208:40:182
3460407993727,6cyclictest0-21swapper/111:50:001
3460407993725,8cyclictest0-21swapper/107:45:251
346040799372,11cyclictest3462034-21idleruntime-cro07:15:021
346040799371,6cyclictest3562613-21head12:00:251
3460407993711,4cyclictest3554551-21/usr/sbin/munin11:40:171
346040799371,11cyclictest3470758-21grep07:35:261
3460407993710,8cyclictest3561348-21basename12:00:011
3460404993724,10cyclictest121ktimers/009:20:140
3460415993629,4cyclictest3472873-21cron07:45:013
3460412993629,4cyclictest3533435-21idleruntime-cro10:40:002
3460412993627,6cyclictest0-21swapper/208:20:012
346040799368,4cyclictest3465586-21unixbench_multi07:20:251
346040799366,8cyclictest3480271-21/usr/sbin/munin08:05:181
3460407993613,4cyclictest231ktimers/110:46:551
3460407993612,5cyclictest231ktimers/110:35:141
3460404993627,5cyclictest3557718-21sh11:50:010
3460404993625,6cyclictest0-21swapper/008:45:230
3460415993527,4cyclictest0-21swapper/310:25:423
3460415993527,4cyclictest0-21swapper/310:25:423
346040799352,9cyclictest3512733-21latency_hist09:40:001
3460407993521,9cyclictest1-21systemd08:20:251
346040799351,9cyclictest3528318-21munin-run10:25:001
3460407993510,4cyclictest231ktimers/109:30:021
346041599349,5cyclictest3552832-21apt-get11:35:123
3460415993426,4cyclictest0-21swapper/311:29:593
3460415993420,11cyclictest3534787-21sshd10:40:583
3460404993427,4cyclictest3469463-21cron07:35:000
3460404993427,4cyclictest0-21swapper/012:00:000
3460404993424,7cyclictest3470690-21grep07:35:250
3460415993326,4cyclictest3521268-21cron10:05:013
3460415993324,6cyclictest3504392-21fschecks_count09:15:163
3460415993321,9cyclictest3542739-21/usr/sbin/munin11:05:183
3460415993320,10cyclictest3475790-21/usr/sbin/munin07:50:253
3460415993317,13cyclictest3535365-21apt-get10:45:153
3460412993323,6cyclictest0-21swapper/208:35:002
3460407993311,5cyclictest231ktimers/110:00:121
3460404993318,12cyclictest0-21swapper/012:24:590
3460415993225,4cyclictest3483335-21apt-get08:15:133
3460415993222,7cyclictest3464590-21apt-get07:20:143
3460415993220,8cyclictest3466741-21memory07:25:203
3460415993219,10cyclictest3510541-21/usr/sbin/munin09:30:263
3460415993218,11cyclictest3480417-21missed_timers08:05:203
3460415993210,5cyclictest3473449-21irqstats07:45:183
3460412993224,4cyclictest0-21swapper/210:50:192
3460412993224,4cyclictest0-21swapper/209:55:002
3460412993224,4cyclictest0-21swapper/207:20:262
3460412993223,6cyclictest0-21swapper/209:05:002
3460412993220,9cyclictest311ktimers/210:45:142
3460407993222,6cyclictest0-21swapper/111:05:251
346040799321,13cyclictest231ktimers/111:55:011
3460407993210,5cyclictest231ktimers/107:30:141
3460404993225,4cyclictest0-21swapper/012:35:240
3460404993225,4cyclictest0-21swapper/011:40:270
3460404993224,6cyclictest3537396-21kworker/u8:1+events_unbound10:50:470
3460404993224,4cyclictest0-21swapper/011:20:000
3460404993223,5cyclictest0-21swapper/008:20:000
3460404993220,9cyclictest438-21cron11:29:590
3460415993125,3cyclictest3530243-21apt-get10:30:153
3460415993123,4cyclictest0-21swapper/309:40:253
3460415993122,4cyclictest0-21swapper/309:55:393
3460415993119,7cyclictest3502527-21apt-get09:10:133
3460415993117,11cyclictest53-21kcompactd011:54:083
3460412993124,4cyclictest0-21swapper/207:15:012
3460412993124,3cyclictest3526380-21kthreadd10:20:002
3460412993121,7cyclictest3545569-21latency_hist11:15:002
3460407993122,6cyclictest0-21swapper/112:25:251
3460407993122,6cyclictest0-21swapper/112:25:251
3460404993123,6cyclictest3477848-21kworker/u8:4+events_unbound09:05:350
3460415993022,4cyclictest0-21swapper/311:55:153
3460415993020,7cyclictest3564572-21sshd12:07:583
3460415993020,7cyclictest3547726-21/usr/sbin/munin11:20:093
3460415993020,7cyclictest3547726-21/usr/sbin/munin11:20:093
3460415993020,7cyclictest3544498-21/usr/sbin/munin11:10:203
3460415993020,7cyclictest3507029-21/usr/sbin/munin09:20:263
3460415993019,8cyclictest3574036-21/usr/sbin/munin12:35:203
3460415993019,8cyclictest3526839-21apt-get10:20:113
3460415993019,8cyclictest3516825-21/usr/sbin/munin09:50:183
3460415993019,7cyclictest0-21swapper/309:00:163
3460415993017,10cyclictest3569433-21/usr/sbin/munin12:20:263
3460415993017,10cyclictest3487315-21/usr/sbin/munin08:25:203
3460415993017,10cyclictest3486432-21cron08:23:593
3460415993014,13cyclictest3512738-21sendmail09:40:013
346041299307,20cyclictest311ktimers/207:37:372
3460412993023,4cyclictest0-21swapper/211:55:262
3460412993023,4cyclictest0-21swapper/211:50:022
3460412993021,6cyclictest3494818-21grep08:45:242
3460407993023,4cyclictest3550905-21kthreadd11:30:001
3460404993023,3cyclictest0-21swapper/007:25:020
3460404993022,5cyclictest3550517-21kworker/u8:1+events_unbound11:50:520
3460404993022,5cyclictest1-21systemd08:20:240
346041599294,19cyclictest0-21swapper/308:55:213
3460415992922,4cyclictest3478237-21apt-get08:00:123
3460415992922,3cyclictest3566684-21apt-get12:15:133
3460415992920,6cyclictest53-21kcompactd010:07:023
3460415992919,7cyclictest3541828-21cron11:01:003
3460415992919,7cyclictest3532473-21/usr/sbin/munin10:35:213
3460415992919,7cyclictest3489745-21sshd08:32:583
3460415992919,7cyclictest3468542-21/usr/sbin/munin07:30:203
3460415992918,8cyclictest3482309-21/usr/sbin/munin08:10:233
3460415992918,8cyclictest3482309-21/usr/sbin/munin08:10:233
3460415992918,8cyclictest3461126-21ntp_states07:10:203
3460415992917,9cyclictest3501244-21/usr/sbin/munin09:05:233
3460415992916,10cyclictest3525029-21diskstats10:15:163
3460415992915,11cyclictest3562212-21/usr/sbin/munin12:00:213
346041299299,4cyclictest311ktimers/212:06:092
346041299298,6cyclictest0-21swapper/209:20:252
346041299297,5cyclictest0-21swapper/208:05:222
346041299296,5cyclictest0-21swapper/207:55:232
346041299295,7cyclictest32-21ksoftirqd/211:50:262
3460412992922,4cyclictest0-21swapper/211:20:232
3460412992922,4cyclictest0-21swapper/211:20:232
3460412992922,3cyclictest0-21swapper/207:50:252
3460412992921,5cyclictest0-21swapper/211:00:012
3460412992921,5cyclictest0-21swapper/207:30:202
3460412992910,10cyclictest3460397-21cyclictest09:15:212
3460407992911,4cyclictest231ktimers/108:55:141
346040499299,5cyclictest131rcu_preempt09:40:000
346040499298,4cyclictest3524202-21awk10:10:250
3460404992922,4cyclictest0-21swapper/011:35:250
3460404992921,5cyclictest0-21swapper/011:05:120
3460415992821,4cyclictest3557721-21sh11:50:013
3460415992819,6cyclictest3571940-21/usr/sbin/munin12:30:163
3460415992818,7cyclictest3490655-21/usr/sbin/munin08:35:183
3460415992818,7cyclictest3476857-21/usr/sbin/munin07:55:183
3460415992816,9cyclictest3551291-21/usr/sbin/munin11:30:163
3460415992814,11cyclictest3540046-21/usr/sbin/munin10:55:273
3460415992811,14cyclictest3570213-21df12:25:153
3460415992811,14cyclictest3570213-21df12:25:143
3460415992811,14cyclictest3508737-21sed09:25:263
346041299289,5cyclictest131rcu_preempt09:08:482
346041299288,6cyclictest32-21ksoftirqd/212:20:232
346041299287,5cyclictest0-21swapper/209:10:162
346041299287,4cyclictest32-21ksoftirqd/211:20:002
3460412992822,3cyclictest0-21swapper/208:25:012
3460412992815,4cyclictest131rcu_preempt10:00:312
3460412992815,4cyclictest131rcu_preempt09:45:142
3460407992819,5cyclictest3531243-21unixbench_multi10:30:281
346040499280,5cyclictest131rcu_preempt12:00:130
3460415992720,4cyclictest3564981-21apt-get12:10:123
3460415992716,8cyclictest3537842-21ntp_states10:50:243
346041299278,5cyclictest0-21swapper/212:00:222
346041299276,6cyclictest0-21swapper/209:59:002
346041299276,6cyclictest0-21swapper/207:15:132
3460412992720,4cyclictest0-21swapper/210:25:262
3460412992720,4cyclictest0-21swapper/210:25:252
3460412992719,4cyclictest0-21swapper/208:35:162
3460412992718,6cyclictest0-21swapper/207:30:012
3460412992717,3cyclictest131rcu_preempt08:04:042
3460412992715,9cyclictest3513330-21cut09:40:192
3460412992715,4cyclictest131rcu_preempt12:16:592
3460412992714,10cyclictest3554552-21/usr/sbin/munin11:40:122
3460412992710,7cyclictest131rcu_preempt08:28:242
3460407992716,7cyclictest3513989-21grep09:40:261
346040499274,5cyclictest131rcu_preempt10:25:120
346040499274,5cyclictest131rcu_preempt10:25:120
3460404992718,6cyclictest0-21swapper/010:10:000
346041599260,5cyclictest0-21swapper/308:50:343
346041299269,8cyclictest131rcu_preempt10:22:062
346041299267,11cyclictest32-21ksoftirqd/212:38:112
3460412992615,8cyclictest3481632-21apt-get08:10:122
3460412992615,8cyclictest3481632-21apt-get08:10:112
3460412992612,8cyclictest3464772-21kworker/u8:1-events_unbound07:46:262
3460412992611,8cyclictest131rcu_preempt07:44:462
346041299260,16cyclictest32-21ksoftirqd/209:27:352
3460404992620,4cyclictest3547124-21kworker/u8:2+events_unbound11:30:520
3460404992620,4cyclictest3441169-21kworker/u8:2+events_unbound07:19:560
3460404992617,6cyclictest3476926-21sort07:55:190
346041299257,5cyclictest32-21ksoftirqd/209:35:192
3460412992519,3cyclictest0-21swapper/211:00:162
3460412992518,3cyclictest0-21swapper/212:30:162
3460412992512,10cyclictest53-21kcompactd011:38:512
346041299251,17cyclictest0-21swapper/212:28:202
346041299251,17cyclictest0-21swapper/212:28:202
346041299251,15cyclictest0-21swapper/211:32:222
346041299251,14cyclictest0-21swapper/210:30:352
346041299250,13cyclictest0-21swapper/212:12:372
3460407992519,3cyclictest0-21swapper/109:05:011
3460407992514,5cyclictest0-21swapper/107:45:011
3460407992513,10cyclictest231ktimers/111:35:121
3460404992521,2cyclictest13550irq/42-30be0000.ethernet12:26:360
3460404992521,2cyclictest13550irq/42-30be0000.ethernet12:26:360
3460404992518,4cyclictest0-21swapper/012:10:230
3460404992518,4cyclictest0-21swapper/008:05:260
3460404992517,6cyclictest121ktimers/009:25:070
3460404992517,4cyclictest0-21swapper/010:05:010
346040499251,6cyclictest3513591-21proc_pri09:40:220
3460404992516,6cyclictest0-21swapper/008:05:000
346040499251,20cyclictest0-21swapper/012:19:140
346040499251,10cyclictest3536840-21idleruntime-cro10:50:000
346041299248,9cyclictest131rcu_preempt10:40:112
346041299241,15cyclictest0-21swapper/208:50:302
3460412992410,4cyclictest32-21ksoftirqd/210:10:182
3460407992418,3cyclictest0-21swapper/110:05:131
3460404992417,4cyclictest0-21swapper/009:55:260
3460404992414,6cyclictest0-21swapper/007:25:250
346040499241,21cyclictest0-21swapper/007:40:260
346040499241,19cyclictest0-21swapper/011:22:170
346040499241,19cyclictest0-21swapper/011:22:160
346041299231,14cyclictest0-21swapper/211:05:232
3460407992314,6cyclictest3476288-21munin-run07:55:011
3460404992317,4cyclictest3489697-21kworker/u8:2+events_unbound09:10:360
3460404992317,3cyclictest0-21swapper/011:15:020
3460404992316,4cyclictest0-21swapper/007:45:170
3460404992313,7cyclictest3539198-21cut10:55:170
346040499231,18cyclictest0-21swapper/010:21:270
346040499231,18cyclictest0-21swapper/008:52:340
346040499231,18cyclictest0-21swapper/007:11:340
346040499230,21cyclictest0-21swapper/009:30:130
3460407992212,7cyclictest231ktimers/107:25:111
3460404992215,3cyclictest0-21swapper/010:32:180
3460404992210,8cyclictest0-21swapper/010:20:010
346040499220,18cyclictest0-21swapper/010:38:490
3460407992115,3cyclictest3540831-21fschecks_count11:00:151
3460404992116,3cyclictest0-21swapper/012:30:160
3460404992116,3cyclictest0-21swapper/009:45:420
3460404992115,4cyclictest0-21swapper/008:10:120
3460404992115,4cyclictest0-21swapper/008:10:120
3460404992115,2cyclictest0-21swapper/009:00:240
3460407992011,6cyclictest0-21swapper/107:55:261
346040799192,12cyclictest3510598-21unixbench_singl09:30:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional