You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-17 - 08:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Wed Jun 17, 2026 00:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3290165995815,24cyclictest3296350-21if_eth019:25:190
3290174995245,4cyclictest3397727-21munin-run00:20:002
3290174994942,4cyclictest3378994-21fschecks_time23:25:162
3290174994924,20cyclictest0-21swapper/223:00:282
3290174994719,5cyclictest3312857-21cron20:15:002
329016599471,26cyclictest3361621-21diskstats22:35:170
3290170994538,5cyclictest3328325-21kworker/u8:0+events_unbound21:11:191
3290165994534,7cyclictest3349326-21sendmail22:00:000
329016599452,23cyclictest3359906-21df22:30:150
329016599451,41cyclictest3400859-21expr00:25:260
329016599451,41cyclictest3359441-21sshd22:28:330
3290174994418,3cyclictest0-21swapper/220:30:322
3290179994336,5cyclictest3399289-21kworker/u8:0+events_unbound00:37:023
3290179994326,13cyclictest391ktimers/321:35:143
3290174994336,4cyclictest3332060-21munin-run21:10:002
3290174994334,6cyclictest3297523-21apt19:30:012
3290170994327,12cyclictest231ktimers/119:20:101
329016599432,34cyclictest3306278-21apt-get19:55:130
329016599431,39cyclictest3396731-21/usr/sbin/munin00:15:210
329016599431,39cyclictest3341702-21/usr/sbin/munin21:35:240
329016599431,37cyclictest3391169-21/usr/sbin/munin00:00:170
329016599431,37cyclictest3302987-21/usr/sbin/munin19:45:150
3290179994233,6cyclictest3333291-21grep21:10:273
3290179994232,6cyclictest0-21swapper/319:46:033
3290165994220,18cyclictest0-21swapper/020:30:010
329016599421,5cyclictest0-21swapper/020:50:140
329016599421,4cyclictest0-21swapper/000:35:110
329016599421,38cyclictest3368507-21/usr/sbin/munin22:55:160
329016599421,2cyclictest0-21swapper/000:05:160
329016599420,4cyclictest0-21swapper/021:10:120
329016599418,24cyclictest0-21swapper/021:15:140
329016599416,29cyclictest3328768-21latency_hist21:00:020
329016599411,4cyclictest0-21swapper/019:15:130
329016599411,3cyclictest3312485-21sshd20:10:330
329016599411,35cyclictest3401585-21/usr/sbin/munin00:30:110
329016599411,35cyclictest3373534-21/usr/sbin/munin23:10:140
329016599411,10cyclictest3335913-21dpkg21:20:060
329016599410,39cyclictest0-21swapper/020:15:170
3290179994033,4cyclictest3356165-21sendmail22:20:013
3290179994029,9cyclictest391ktimers/321:30:133
3290174994016,6cyclictest311ktimers/222:20:012
3290170994034,4cyclictest3361114-21kworker/u8:1+events_unbound22:46:381
3290170994034,4cyclictest3302589-21kworker/u8:2+events_unbound19:51:071
3290170994032,6cyclictest3347584-21kworker/u8:1+events_unbound22:21:321
329017099401,14cyclictest24-21ksoftirqd/122:00:121
329016599402,30cyclictest3370045-21apt-get23:00:130
329016599401,5cyclictest3350203-21/usr/sbin/munin22:00:240
3290165994015,22cyclictest121ktimers/020:20:120
329016599401,36cyclictest3375983-21/usr/sbin/munin23:15:240
329016599401,34cyclictest3342863-21diskstats21:40:140
329016599401,34cyclictest3339327-21apt-get21:30:130
329016599401,34cyclictest3290322-21/usr/sbin/munin19:10:130
329016599401,33cyclictest3344446-21/usr/sbin/munin21:45:110
329016599401,33cyclictest3344446-21/usr/sbin/munin21:45:110
329016599401,33cyclictest3294349-21apt-get19:20:100
329016599401,23cyclictest3301126-21wc19:40:010
329016599400,2cyclictest0-21swapper/019:40:120
3290179993933,4cyclictest3397758-21idleruntime-cro00:20:003
3290174993932,4cyclictest3388769-21sh23:55:002
329016599398,27cyclictest3390055-21grep23:55:270
329016599393,33cyclictest3321665-21latency_hist20:40:010
329016599392,34cyclictest3357143-21/usr/sbin/munin22:20:240
329016599391,4cyclictest0-21swapper/020:45:080
329016599391,3cyclictest3382182-21apt-get23:35:110
329016599391,35cyclictest3331305-21mailstats21:05:240
329016599391,33cyclictest3346262-21df_inode21:50:150
329016599391,33cyclictest3346262-21df_inode21:50:150
329016599391,33cyclictest3308505-21sed20:00:200
329016599391,33cyclictest3308505-21sed20:00:200
329016599390,4cyclictest3330395-21latency_hist21:05:010
329016599390,4cyclictest0-21swapper/022:05:160
3290174993827,7cyclictest0-21swapper/220:00:002
3290170993828,5cyclictest0-21swapper/120:30:161
3290170993815,12cyclictest131rcu_preempt20:10:001
329016599383,32cyclictest3386051-21meminfo23:45:190
329016599382,33cyclictest3338499-21/usr/sbin/munin21:25:270
329016599381,35cyclictest3399449-21sshd00:22:360
329016599381,34cyclictest3394395-21/usr/sbin/munin00:10:130
329016599381,34cyclictest3384345-21/usr/sbin/munin23:40:180
329016599381,34cyclictest3380533-21/usr/sbin/munin23:30:130
329016599381,34cyclictest3365445-21/usr/sbin/munin22:45:210
329016599381,34cyclictest3355239-21/usr/sbin/munin22:15:230
329016599381,34cyclictest3323230-21cron20:44:000
329016599381,34cyclictest3319281-21/usr/sbin/munin20:30:310
329016599381,32cyclictest629-21snmpd22:50:150
329016599381,32cyclictest3371745-21/usr/sbin/munin23:05:130
3290165993812,23cyclictest3354443-21latency_hist22:15:010
3290165993811,23cyclictest3309844-21cut20:05:180
329016599380,4cyclictest3378899-21diskstats23:25:150
3290165993728,4cyclictest0-21swapper/022:41:360
329016599372,31cyclictest0-21swapper/019:50:150
329016599371,3cyclictest0-21swapper/023:50:120
329016599371,33cyclictest3377420-21/usr/sbin/munin23:20:200
329016599371,33cyclictest3298275-21/usr/sbin/munin19:30:210
329016599371,31cyclictest3290250-21sed19:10:020
3290179993629,4cyclictest3295824-21kthreadd19:25:013
3290174993626,7cyclictest0-21swapper/200:40:012
329017099362,5cyclictest0-21swapper/120:25:121
3290179993524,7cyclictest3318108-21latency_hist20:30:013
3290174993527,4cyclictest0-21swapper/221:45:162
3290174993527,4cyclictest0-21swapper/221:45:162
3290170993528,5cyclictest3379964-21kworker/u8:2+events_unbound23:38:401
3290170993526,4cyclictest0-21swapper/123:11:461
3290170993525,6cyclictest0-21swapper/120:40:001
3290170993524,7cyclictest0-21swapper/100:15:271
3290170993523,3cyclictest131rcu_preempt00:10:151
3290179993427,5cyclictest3307473-21kworker/u8:0+events_unbound20:06:083
3290174993423,7cyclictest3374483-21grep23:10:262
3290170993428,4cyclictest3377976-21kworker/u8:0+events_unbound23:28:421
3290170993428,4cyclictest3302589-21kworker/u8:2+events_unbound19:46:041
3290170993417,5cyclictest131rcu_preempt19:20:011
3290179993324,5cyclictest0-21swapper/320:45:003
3290174993325,4cyclictest0-21swapper/221:40:192
3290174993325,4cyclictest0-21swapper/220:45:012
3290174993325,4cyclictest0-21swapper/220:35:162
3290174993322,7cyclictest0-21swapper/221:55:012
3290174993322,7cyclictest0-21swapper/221:55:002
3290170993322,8cyclictest0-21swapper/122:40:001
3290170993322,5cyclictest131rcu_preempt23:45:251
3290170993313,5cyclictest131rcu_preempt19:30:131
3290170993313,12cyclictest131rcu_preempt21:45:211
3290170993313,12cyclictest131rcu_preempt21:45:211
3290179993224,4cyclictest0-21swapper/322:50:003
3290179993224,4cyclictest0-21swapper/320:30:323
3290174993224,4cyclictest0-21swapper/223:40:252
3290174993223,4cyclictest0-21swapper/222:31:332
3290174993221,7cyclictest3351432-21tune2fs22:05:182
3290170993226,4cyclictest3335151-21kworker/u8:2+events_unbound21:21:231
3290170993225,5cyclictest3321581-21kworker/u8:2+events_unbound20:41:151
3290170993224,4cyclictest0-21swapper/123:00:281
3290170993223,4cyclictest0-21swapper/121:16:201
3290170993221,7cyclictest0-21swapper/121:05:001
3290170993220,5cyclictest131rcu_preempt21:35:261
3290179993124,5cyclictest3364557-21kworker/u8:0+events_unbound22:51:423
3290179993123,6cyclictest3377976-21kworker/u8:0+events_unbound23:21:503
3290179993123,4cyclictest0-21swapper/322:40:003
3290179993122,5cyclictest0-21swapper/300:00:003
3290174993122,5cyclictest3373209-21sh23:10:002
3290174993120,7cyclictest3354441-21sh22:15:012
329017099318,12cyclictest131rcu_preempt00:25:001
329017099313,6cyclictest3298525-21kworker/u8:0+writeback19:40:011
3290170993125,4cyclictest3361114-21kworker/u8:1+events_unbound22:41:361
3290170993123,5cyclictest3385370-21sh23:45:001
3290170993113,10cyclictest131rcu_preempt20:46:141
3290170993112,11cyclictest24-21ksoftirqd/100:40:001
3290170993112,11cyclictest131rcu_preempt23:20:131
3290170993111,12cyclictest24-21ksoftirqd/122:25:211
3290170993110,17cyclictest231ktimers/122:58:031
3290170993110,12cyclictest131rcu_preempt23:50:221
329017999303,5cyclictest131rcu_preempt23:35:133
3290179993024,4cyclictest3374896-21kworker/u8:1+events_unbound23:16:473
3290179993023,3cyclictest0-21swapper/319:20:013
3290174993023,4cyclictest0-21swapper/223:30:242
329017099309,13cyclictest131rcu_preempt23:55:251
329017099308,12cyclictest131rcu_preempt21:55:001
329017099308,12cyclictest131rcu_preempt21:55:001
3290170993023,5cyclictest3328325-21kworker/u8:0+events_unbound21:09:401
3290170993022,4cyclictest0-21swapper/122:15:281
3290170993022,4cyclictest0-21swapper/120:00:001
3290170993022,4cyclictest0-21swapper/100:30:161
3290170993017,5cyclictest131rcu_preempt21:40:161
3290170993016,6cyclictest131rcu_preempt21:30:021
3290170993014,5cyclictest131rcu_preempt22:05:211
3290170993011,11cyclictest131rcu_preempt20:15:151
3290170993010,12cyclictest131rcu_preempt19:10:251
3290170993010,10cyclictest0-21swapper/100:02:261
3290179992920,4cyclictest0-21swapper/321:55:003
3290179992920,4cyclictest0-21swapper/321:55:003
3290174992922,3cyclictest0-21swapper/222:55:162
3290174992922,3cyclictest0-21swapper/220:30:012
3290174992921,4cyclictest0-21swapper/222:50:272
3290174992921,4cyclictest0-21swapper/220:50:172
3290174992920,4cyclictest0-21swapper/220:05:182
329017099299,5cyclictest131rcu_preempt22:10:201
3290170992910,11cyclictest131rcu_preempt19:30:011
3290179992819,6cyclictest3332051-21sh21:10:003
3290179992819,6cyclictest3292070-21ls19:15:033
3290179992818,7cyclictest3364214-21unixbench_multi22:40:293
3290174992820,6cyclictest3324115-21kernelversion20:45:212
3290174992819,6cyclictest0-21swapper/221:00:002
3290174992819,6cyclictest0-21swapper/219:50:002
329017099289,12cyclictest24-21ksoftirqd/119:40:241
329017099288,11cyclictest24-21ksoftirqd/120:00:171
329017099288,11cyclictest24-21ksoftirqd/120:00:171
329017099288,11cyclictest131rcu_preempt22:50:211
329017099286,11cyclictest24-21ksoftirqd/123:35:011
3290170992820,4cyclictest3311608-21grep20:10:171
3290170992818,7cyclictest0-21swapper/121:55:251
3290170992812,3cyclictest131rcu_preempt21:30:221
3290170992811,9cyclictest131rcu_preempt00:06:441
3290170992810,11cyclictest131rcu_preempt20:55:211
329017099280,8cyclictest0-21swapper/123:05:131
329017999278,5cyclictest131rcu_preempt23:45:253
3290179992721,4cyclictest3347584-21kworker/u8:1+events_unbound22:06:293
3290179992721,4cyclictest3345797-21kworker/u8:0+events_unbound22:01:293
3290179992720,3cyclictest0-21swapper/323:05:243
3290179992719,4cyclictest0-21swapper/322:55:253
3290179992718,6cyclictest3312860-21sh20:15:003
3290179992717,7cyclictest3345907-21cron21:50:003
3290179992717,7cyclictest3345907-21cron21:50:003
3290179992716,7cyclictest0-21swapper/321:20:273
3290174992719,5cyclictest0-21swapper/222:45:262
3290174992719,4cyclictest0-21swapper/221:40:002
3290174992719,4cyclictest0-21swapper/200:00:252
329017099276,6cyclictest24-21ksoftirqd/120:50:171
3290179992619,4cyclictest0-21swapper/323:00:183
3290179992619,3cyclictest0-21swapper/323:50:153
3290174992620,3cyclictest0-21swapper/221:17:002
3290174992619,4cyclictest3291788-21sh19:15:012
3290174992619,3cyclictest0-21swapper/223:20:152
3290174992618,5cyclictest0-21swapper/200:10:002
3290174992618,4cyclictest0-21swapper/222:00:272
3290174992617,4cyclictest0-21swapper/220:15:152
329017099268,4cyclictest131rcu_preempt00:25:211
329017099268,3cyclictest131rcu_preempt20:20:241
3290170992610,4cyclictest131rcu_preempt23:15:151
3290179992518,4cyclictest0-21swapper/321:55:173
3290179992518,3cyclictest0-21swapper/323:26:493
3290179992518,3cyclictest0-21swapper/320:55:243
3290179992517,4cyclictest0-21swapper/322:10:133
3290174992519,3cyclictest0-21swapper/221:25:012
3290174992518,4cyclictest0-21swapper/200:10:252
3290174992518,3cyclictest0-21swapper/220:00:232
3290174992518,3cyclictest0-21swapper/220:00:232
3290174992517,6cyclictest311ktimers/223:45:132
3290174992515,7cyclictest3362073-21ntp_states22:35:232
3290170992512,5cyclictest131rcu_preempt22:30:121
329017999242,5cyclictest131rcu_preempt00:10:123
3290179992418,3cyclictest391ktimers/320:15:123
3290179992416,5cyclictest391ktimers/319:35:253
3290179992416,4cyclictest0-21swapper/300:01:553
3290179992415,5cyclictest3357866-21audit_send_reply22:25:013
3290174992418,3cyclictest0-21swapper/219:50:192
3290174992417,4cyclictest0-21swapper/219:25:012
3290174992417,3cyclictest0-21swapper/219:35:252
3290174992416,5cyclictest0-21swapper/223:15:162
3290174992416,4cyclictest0-21swapper/219:30:162
3290174992415,5cyclictest0-21swapper/221:30:272
329017999231,7cyclictest3339099-21munin-run21:30:013
3290179992315,4cyclictest0-21swapper/320:01:063
3290179992315,4cyclictest0-21swapper/320:01:063
329017999231,4cyclictest131rcu_preempt19:30:133
3290179992314,5cyclictest3361261-21cron22:35:003
3290179992313,6cyclictest3297041-21grep19:25:273
3290179992313,6cyclictest0-21swapper/300:30:003
3290174992317,3cyclictest0-21swapper/219:40:252
3290174992316,4cyclictest3348033-21fschecks_time21:55:162
3290179992216,4cyclictest391ktimers/321:40:203
329017999221,5cyclictest3325653-21fschecks_time20:50:173
3290179992215,3cyclictest0-21swapper/320:35:263
3290179992214,4cyclictest0-21swapper/323:10:263
3290179992213,6cyclictest3306378-21ls19:55:143
3290179992213,5cyclictest3393464-21head00:05:293
3290174992215,4cyclictest0-21swapper/221:05:002
3290174992212,7cyclictest3292296-21cat19:15:032
329017999215,4cyclictest131rcu_preempt19:40:233
3290179992115,3cyclictest3335574-21latency_hist21:20:013
3290179992114,4cyclictest3379964-21kworker/u8:2+events_unbound23:40:163
3290179992114,4cyclictest3359271-21sshd22:26:333
3290179992113,5cyclictest391ktimers/320:23:133
3290179992113,5cyclictest0-21swapper/300:20:273
3290179992111,7cyclictest0-21swapper/319:55:003
3290174992115,3cyclictest0-21swapper/221:25:252
3290174992115,3cyclictest0-21swapper/200:25:242
3290174992114,4cyclictest3389200-21fschecks_time23:55:162
3290174992113,6cyclictest3358185-21awk22:25:152
3290174992111,7cyclictest204-21dbus-daemon21:11:192
3290179992014,4cyclictest3323763-21apt20:45:113
3290179992013,4cyclictest3381472-21tail23:30:243
329017999201,17cyclictest3328886-21seq21:00:103
3290174992016,2cyclictest0-21swapper/222:20:162
3290174992014,3cyclictest0-21swapper/222:40:232
3290174992010,7cyclictest3316041-21seq20:22:052
329017999190,5cyclictest0-21swapper/300:30:313
3290174991914,3cyclictest3382905-21sendmail_mailtr23:35:232
329017499190,4cyclictest0-21swapper/200:30:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional