You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-15 - 07:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Fri May 15, 2026 00:44:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3459299995426,25cyclictest3574541-21sendmail00:40:012
3459303995339,6cyclictest3497518-21kworker/u8:1+events_unbound21:07:463
3459294995325,24cyclictest3567520-21basename00:20:001
345929499512,15cyclictest3542984-21idleruntime-cro23:10:001
345929499502,16cyclictest3547874-21unixbench_singl23:20:261
345929499502,16cyclictest3547874-21unixbench_singl23:20:251
3459286995016,20cyclictest0-21swapper/023:50:000
3459286994938,6cyclictest3484119-21sh20:19:590
3459299994828,15cyclictest0-21swapper/221:55:002
345929499482,14cyclictest3499481-21cron21:05:001
3459303994733,10cyclictest391ktimers/319:50:123
3459303994732,12cyclictest391ktimers/319:40:133
3459299994728,15cyclictest0-21swapper/223:00:282
3459299994727,16cyclictest0-21swapper/220:00:152
3459299994724,19cyclictest0-21swapper/220:30:252
3459299994724,19cyclictest0-21swapper/220:30:252
3459299994717,26cyclictest3469243-21meminfo19:35:192
3459294994735,8cyclictest3539582-21sendmail23:00:001
3459294994726,18cyclictest3560660-21kthreadd00:00:001
3459299994621,21cyclictest0-21swapper/220:20:002
3459299994617,25cyclictest0-21swapper/221:25:142
3459294994622,20cyclictest0-21swapper/122:25:191
345929499462,14cyclictest3553140-21tail23:35:261
345929499462,12cyclictest3510591-21fschecks_time21:35:171
3459299994524,17cyclictest0-21swapper/200:00:292
345929499454,14cyclictest3535736-21unixbench_singl22:45:281
3459294994514,5cyclictest3475383-21sh19:55:011
3459303994429,12cyclictest391ktimers/322:30:133
3459303994329,11cyclictest391ktimers/323:45:123
3459299994332,6cyclictest3539581-21sendmail23:00:002
3459299994326,13cyclictest0-21swapper/220:20:242
3459299994326,13cyclictest0-21swapper/220:20:232
3459299994322,17cyclictest0-21swapper/222:35:282
345929999431,9cyclictest3545639-21/usr/sbin/munin23:15:202
3459294994318,3cyclictest0-21swapper/122:00:161
3459294994317,4cyclictest231ktimers/119:40:161
3459286994331,9cyclictest121ktimers/021:30:130
345929999421,8cyclictest3500384-21/usr/sbin/munin21:05:192
345929999421,8cyclictest3489477-21/usr/sbin/munin20:35:282
345929999421,8cyclictest3476300-21/usr/sbin/munin19:55:232
345929999421,8cyclictest3463480-21munin-node19:20:102
345929999421,6cyclictest3523489-21/usr/sbin/munin22:10:242
345929999421,6cyclictest3523489-21/usr/sbin/munin22:10:242
345929999421,12cyclictest3504848-21idleruntime-cro21:20:002
3459294994233,6cyclictest3497776-21cron21:00:001
3459294994229,8cyclictest0-21swapper/122:45:011
3459286994233,6cyclictest0-21swapper/019:15:010
3459286994212,10cyclictest0-21swapper/021:40:250
3459299994129,8cyclictest3463252-21sendmail19:20:012
345929999411,9cyclictest3566248-21/usr/sbin/munin00:15:162
345929999411,9cyclictest3502846-21cron21:11:592
3459299994112,26cyclictest3509026-21cut21:30:192
345929999411,11cyclictest3558107-21ps23:50:232
345929999411,10cyclictest3467658-21ntp_states19:30:202
3459294994133,5cyclictest3574075-21sed00:35:281
3459286994130,8cyclictest3517177-21auditctl21:55:000
345929999407,7cyclictest3496127-21date20:55:012
3459299994033,4cyclictest3522514-21latency_hist22:10:002
3459299994033,4cyclictest3522514-21latency_hist22:10:002
3459299994030,6cyclictest0-21swapper/222:20:002
3459299994030,6cyclictest0-21swapper/222:20:002
345929999402,7cyclictest53-21kcompactd023:22:502
345929999402,7cyclictest53-21kcompactd023:22:502
3459299994024,12cyclictest0-21swapper/223:30:282
3459299994019,18cyclictest0-21swapper/221:40:012
345929999401,8cyclictest3528202-21/usr/sbin/munin22:25:162
3459299994018,18cyclictest0-21swapper/200:20:112
3459299994015,21cyclictest3492183-21grep20:40:262
345929999401,11cyclictest3519700-21ntp_states22:00:212
3459294994015,3cyclictest0-21swapper/121:15:281
3459303993931,6cyclictest3502728-21kworker/u8:0+events_unbound21:56:423
3459303993930,6cyclictest3501828-21fschecks_time21:10:173
345929999393,5cyclictest3556355-21ntp_states23:45:222
345929999392,9cyclictest3480502-21cron20:10:012
345929999391,9cyclictest3505725-21memory21:20:202
345929999391,7cyclictest3481579-21/usr/sbin/munin20:10:222
345929999391,6cyclictest3553138-21grep23:35:262
345929999391,6cyclictest35378612latency_hist22:55:012
3459299993916,19cyclictest0-21swapper/221:45:012
3459299993913,23cyclictest0-21swapper/222:20:132
345929999391,11cyclictest3562952-21irqstats00:05:202
345929999391,10cyclictest3465268-21sort19:25:132
3459294993931,6cyclictest3497518-21kworker/u8:1+events_unbound21:22:491
3459294993928,7cyclictest0-21swapper/120:35:271
3459294993926,9cyclictest231ktimers/121:30:121
3459286993930,6cyclictest3501415-21sh21:10:000
3459303993828,6cyclictest3560229-21unixbench_singl23:55:273
3459303993822,9cyclictest131rcu_preempt19:17:523
345929999388,9cyclictest3569736-21awk00:25:142
345929999387,4cyclictest0-21swapper/222:45:172
345929999386,6cyclictest3559795-21proc_pri23:55:242
345929999383,6cyclictest3548990-21meminfo23:25:192
345929999383,6cyclictest3548990-21meminfo23:25:192
345929999381,9cyclictest3518484-21users21:55:262
345929999381,8cyclictest3530183-21/usr/sbin/munin22:30:222
345929999381,8cyclictest3498399-21grep21:00:192
345929999381,7cyclictest3474144-21fschecks_time19:50:162
3459299993814,21cyclictest3532983-21/usr/sbin/munin22:40:212
345929999381,11cyclictest3571923-21ntp_states00:30:232
345929999381,11cyclictest3473218-21grep19:45:252
345929999381,10cyclictest3515088-21users21:45:282
3459286993831,4cyclictest3532738-21kthreadd22:40:010
3459303993730,5cyclictest3530913-21kworker/u8:0+events_unbound22:54:413
3459303993726,6cyclictest0-21swapper/319:30:243
3459303993724,4cyclictest131rcu_preempt23:15:323
3459303993723,9cyclictest131rcu_preempt20:34:213
3459303993723,9cyclictest131rcu_preempt20:34:213
3459303993720,4cyclictest131rcu_preempt22:57:243
3459303993720,4cyclictest0-21swapper/321:42:293
345929999375,5cyclictest3486290-21fschecks_time20:25:142
345929999372,9cyclictest3544270-21grep23:10:282
345929999372,5cyclictest3459458-21apt-get19:10:122
345929999372,4cyclictest3564418-21df00:10:142
345929999371,10cyclictest629-21snmpd23:05:192
345930399366,7cyclictest40-21ksoftirqd/323:40:003
3459303993629,4cyclictest3484150-21sendmail20:20:003
3459303993626,6cyclictest3574113-21unixbench_singl00:35:293
3459303993618,10cyclictest131rcu_preempt19:20:143
3459303993616,12cyclictest0-21swapper/319:46:113
3459303993615,4cyclictest0-21swapper/320:05:193
3459303993615,13cyclictest131rcu_preempt23:40:233
345929999363,6cyclictest3493350-21memory20:45:192
345929999361,8cyclictest3553991-21diskstats23:40:152
345929999361,7cyclictest3471088-21/usr/sbin/munin19:40:212
3459294993625,7cyclictest3468097-21grep19:30:241
345928699360,21cyclictest3491134-21latency_hist20:40:010
3459303993524,7cyclictest0-21swapper/323:14:593
3459303993524,4cyclictest131rcu_preempt22:06:233
3459303993524,4cyclictest131rcu_preempt22:06:233
3459303993524,4cyclictest131rcu_preempt21:18:233
3459303993519,4cyclictest131rcu_preempt20:55:173
3459303993518,10cyclictest131rcu_preempt21:35:183
3459303993518,10cyclictest131rcu_preempt20:10:163
3459303993517,3cyclictest0-21swapper/320:36:183
3459303993517,3cyclictest0-21swapper/300:01:053
3459303993515,17cyclictest391ktimers/323:07:063
3459303993513,11cyclictest131rcu_preempt22:16:193
3459303993513,11cyclictest131rcu_preempt22:16:193
345929999351,8cyclictest3496356-21/usr/sbin/munin20:55:132
3459286993511,6cyclictest11-21ksoftirqd/021:35:240
3459303993428,4cyclictest3463048-21kworker/u8:2+events_unbound19:27:253
3459303993423,4cyclictest131rcu_preempt23:50:333
3459303993417,9cyclictest131rcu_preempt22:48:323
3459303993417,4cyclictest131rcu_preempt00:29:063
3459303993413,13cyclictest131rcu_preempt20:25:203
3459303993411,5cyclictest131rcu_preempt23:34:403
345928699347,9cyclictest3506016-21grep21:20:240
345928699345,10cyclictest3567684-21cat00:20:010
345930399338,16cyclictest40-21ksoftirqd/319:40:013
3459303993323,4cyclictest131rcu_preempt21:35:003
3459303993322,6cyclictest0-21swapper/322:10:263
3459303993322,6cyclictest0-21swapper/322:10:263
3459303993322,6cyclictest0-21swapper/321:25:273
3459303993321,5cyclictest3511650-21kworker/u8:3+events_unbound21:53:213
3459303993320,4cyclictest131rcu_preempt19:12:023
3459303993318,7cyclictest131rcu_preempt21:20:013
3459303993316,4cyclictest131rcu_preempt20:40:153
3459303993315,4cyclictest131rcu_preempt21:02:443
3459303993314,9cyclictest131rcu_preempt20:46:013
3459303993314,4cyclictest131rcu_preempt23:27:503
3459303993314,4cyclictest131rcu_preempt23:27:503
3459303993312,5cyclictest0-21swapper/322:04:543
3459303993311,11cyclictest0-21swapper/320:24:503
3459303993311,11cyclictest0-21swapper/320:24:503
3459303993310,12cyclictest0-21swapper/323:22:253
3459303993310,12cyclictest0-21swapper/323:22:253
3459294993322,7cyclictest3516753-21grep21:50:261
345928699338,5cyclictest11-21ksoftirqd/023:20:160
345928699338,5cyclictest11-21ksoftirqd/023:20:160
3459286993326,4cyclictest3480553-21latency_hist20:10:010
3459286993323,7cyclictest3518896-21sendmail22:00:000
3459286993322,8cyclictest13550irq/42-30be0000.ethernet00:20:110
3459303993227,3cyclictest3514575-21ntp_states21:45:243
3459303993226,4cyclictest3516910-21kworker/u8:2+events_unbound22:29:413
3459303993224,5cyclictest0-21swapper/300:10:163
3459303993219,4cyclictest131rcu_preempt23:01:503
3459303993217,4cyclictest131rcu_preempt19:56:103
3459303993216,9cyclictest131rcu_preempt22:20:133
3459303993216,4cyclictest131rcu_preempt22:36:453
3459303993214,9cyclictest131rcu_preempt00:22:263
3459303993214,4cyclictest131rcu_preempt00:05:183
3459303993212,4cyclictest131rcu_preempt22:43:313
3459294993224,4cyclictest0-21swapper/122:35:271
345928699329,4cyclictest11-21ksoftirqd/022:20:230
3459286993223,5cyclictest3514225-21fschecks_time21:45:170
345928699322,13cyclictest3553623-21latency_hist23:40:010
3459286993212,5cyclictest3478809-21sh20:05:010
3459303993117,4cyclictest131rcu_preempt20:03:143
3459303993116,4cyclictest131rcu_preempt20:51:003
3459303993116,3cyclictest0-21swapper/300:30:573
3459294993123,5cyclictest3459279-21cyclictest20:17:271
3459294993121,5cyclictest0-21swapper/119:35:151
345928699316,9cyclictest3560818-21latency_hist00:00:000
345928699315,9cyclictest3524998-21ntpq22:15:210
345928699315,9cyclictest3524998-21ntpq22:15:200
3459286993124,3cyclictest0-21swapper/020:35:000
3459286993124,3cyclictest0-21swapper/020:35:000
3459286993122,6cyclictest3539146-21unixbench_singl22:55:250
3459286993122,6cyclictest0-21swapper/022:15:000
3459286993122,6cyclictest0-21swapper/022:15:000
345930399308,11cyclictest40-21ksoftirqd/300:15:253
345929499306,5cyclictest131rcu_preempt20:15:001
3459294993023,4cyclictest0-21swapper/122:50:271
3459294993023,4cyclictest0-21swapper/100:20:161
3459294993023,3cyclictest0-21swapper/119:15:011
3459294993020,6cyclictest0-21swapper/120:00:251
3459294993019,7cyclictest3572813-21sh00:35:001
3459294993018,7cyclictest0-21swapper/119:55:251
345928699302,12cyclictest3487323-21sshd20:27:260
3459286993016,7cyclictest131rcu_preempt22:09:200
3459286993016,7cyclictest131rcu_preempt22:09:200
3459286993015,9cyclictest3534676-21kworker/u8:2-events_freezable_power_22:47:050
3459294992921,4cyclictest0-21swapper/123:10:291
3459294992920,6cyclictest0-21swapper/122:15:151
3459294992920,6cyclictest0-21swapper/122:15:151
3459294992920,6cyclictest0-21swapper/120:50:241
3459294992920,5cyclictest0-21swapper/120:30:001
3459294992919,7cyclictest3554921-21grep23:40:261
345928699298,7cyclictest11-21ksoftirqd/022:25:100
345928699298,4cyclictest0-21swapper/000:35:220
3459286992920,7cyclictest3504821-21munin-run21:20:000
3459286992920,6cyclictest3501804-21fschecks_count21:10:170
3459286992920,6cyclictest0-21swapper/019:45:010
3459286992920,5cyclictest0-21swapper/023:35:000
345928699291,25cyclictest0-21swapper/023:15:120
3459294992822,3cyclictest3558968-21idleruntime-cro23:55:001
3459294992820,5cyclictest3529747-21fschecks_time22:30:151
3459294992819,6cyclictest3570688-21grep00:25:271
3459294992819,6cyclictest231ktimers/121:05:071
345928699287,4cyclictest0-21swapper/021:02:190
3459286992823,2cyclictest13550irq/42-30be0000.ethernet23:10:280
3459286992819,6cyclictest0-21swapper/019:20:010
3459286992818,6cyclictest0-21swapper/023:05:250
345928699281,25cyclictest0-21swapper/023:40:150
345928699281,13cyclictest0-21swapper/020:20:220
345928699281,13cyclictest0-21swapper/020:20:220
3459294992721,3cyclictest0-21swapper/120:20:151
3459294992721,3cyclictest0-21swapper/120:20:151
3459294992720,4cyclictest0-21swapper/121:45:001
3459294992718,5cyclictest3540854-21grep23:00:271
345928699277,6cyclictest0-21swapper/020:55:190
345928699276,7cyclictest3477203-21date20:00:010
345928699276,5cyclictest11-21ksoftirqd/023:50:170
3459286992712,9cyclictest53-21kcompactd019:50:240
3459294992618,5cyclictest0-21swapper/121:45:281
3459294992616,8cyclictest231ktimers/120:05:131
3459294992613,8cyclictest0-21swapper/123:45:131
345928699265,5cyclictest0-21swapper/020:13:090
3459286992617,6cyclictest3561049-21fschecks_count00:00:140
3459286992616,3cyclictest131rcu_preempt23:25:390
3459286992616,3cyclictest131rcu_preempt23:25:380
3459286992615,7cyclictest3520621-21http22:05:010
3459286992613,10cyclictest3570322-21/usr/sbin/munin00:25:240
345928699260,6cyclictest0-21swapper/020:45:220
3459294992516,5cyclictest0-21swapper/122:00:001
3459294992516,5cyclictest0-21swapper/119:15:251
3459294992515,7cyclictest3564720-21grep00:10:191
345928699257,14cyclictest3466193-21grep19:25:240
3459286992519,3cyclictest0-21swapper/019:35:240
3459286992518,4cyclictest0-21swapper/019:20:250
3459286992517,4cyclictest0-21swapper/019:30:240
3459286992516,6cyclictest3565327-21unixbench_multi00:10:260
3459286992515,6cyclictest0-21swapper/020:40:260
3459286992511,7cyclictest131rcu_preempt22:34:310
345928699251,14cyclictest0-21swapper/000:07:330
3459286992510,9cyclictest131rcu_preempt19:47:420
345929499247,11cyclictest3526341-21grep22:20:141
345929499242,17cyclictest3502654-21sed21:10:271
3459294992418,3cyclictest0-21swapper/123:25:251
3459294992418,3cyclictest0-21swapper/123:25:251
3459294992418,3cyclictest0-21swapper/100:05:191
3459294992417,4cyclictest3546185-21head23:15:261
3459294992417,4cyclictest0-21swapper/122:05:251
3459294992417,4cyclictest0-21swapper/122:05:251
3459294992417,3cyclictest0-21swapper/119:25:151
3459286992417,5cyclictest0-21swapper/022:40:260
3459286992417,3cyclictest0-21swapper/000:34:110
3459286992411,8cyclictest131rcu_preempt21:29:230
345928699240,20cyclictest0-21swapper/020:52:300
3459294992316,4cyclictest0-21swapper/120:40:261
3459294992314,6cyclictest3488180-21cat20:30:181
3459294992314,6cyclictest3488180-21cat20:30:181
345928699238,4cyclictest131rcu_preempt23:00:170
3459286992317,4cyclictest13550irq/42-30be0000.ethernet22:50:170
345929499212,14cyclictest3508437-21cron21:30:011
345929499212,13cyclictest3464515-21grep19:20:251
3459294992114,4cyclictest231ktimers/123:30:431
3459294992114,4cyclictest231ktimers/119:45:371
3459294992113,5cyclictest231ktimers/100:00:141
3459294992112,6cyclictest231ktimers/120:45:131
345929499202,13cyclictest3524198-21cron22:15:001
345929499202,13cyclictest3524198-21cron22:14:591
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional