You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-17 - 18:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot5s.osadl.org (updated Mon Aug 17, 2026 12:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3479858995548,4cyclictest3499372-21munin-plugin-st08:05:000
3479871995439,10cyclictest391ktimers/310:45:123
3479858995123,24cyclictest3588655-21grep12:20:250
3479871995026,20cyclictest0-21swapper/311:30:163
3479871995026,20cyclictest0-21swapper/311:30:153
3479858995040,6cyclictest3561602-21cron11:05:000
3479871994935,10cyclictest391ktimers/312:30:113
3479871994935,10cyclictest391ktimers/312:30:113
3479867994842,4cyclictest3488901-21kworker/u8:3+events_unbound07:45:012
3479858994741,4cyclictest3537522-21munin-plugin-st09:55:000
3479858994622,19cyclictest3504409-21grep08:15:270
3479871994537,5cyclictest3523868-21idleruntime-cro09:15:003
3479871994419,20cyclictest0-21swapper/312:20:253
3479871994334,6cyclictest3490642-21sendmail07:40:003
3479871994334,6cyclictest3490642-21sendmail07:40:003
3479871994312,28cyclictest3553500-21tune2fs10:40:173
3479871994217,21cyclictest3545186-21grep10:15:183
3479862994234,6cyclictest3482320-21kworker/u8:3+events_unbound07:24:541
347985899429,30cyclictest3500938-21kworker/u8:2+events_unbound08:10:260
3479871994134,4cyclictest3573557-21sendmail11:39:593
3479871994134,4cyclictest3573557-21sendmail11:39:593
347987199411,10cyclictest3533095-21meminfo09:40:243
3479867994134,5cyclictest3592196-21kworker/u8:2+events_unbound12:36:112
3479867994134,5cyclictest3592196-21kworker/u8:2+events_unbound12:36:102
347985899417,31cyclictest3555077-21/usr/sbin/munin10:45:140
3479858994130,7cyclictest3592100-21grep12:30:250
3479858994130,7cyclictest3592100-21grep12:30:240
347987199409,27cyclictest3481005-21grep07:10:263
3479871994015,21cyclictest0-21swapper/308:15:273
347987199401,10cyclictest3537061-21tail09:50:233
347987199401,10cyclictest3493507-21sed07:45:273
3479862994030,7cyclictest3550954-21grep10:30:271
3479858994031,6cyclictest3526865-21unixbench_singl09:20:240
3479858994031,6cyclictest3526865-21unixbench_singl09:20:240
347985899401,9cyclictest3531050-21/usr/sbin/munin09:35:170
347985899401,5cyclictest3494452-21grep07:50:170
347985899401,2cyclictest0-21swapper/008:35:120
347985899401,10cyclictest3561844-21apt-get11:05:120
347985899401,10cyclictest3532767-21diskstats09:40:200
3479871993931,4cyclictest0-21swapper/308:00:013
347987199392,8cyclictest208-21systemd-logind09:05:263
347987199392,10cyclictest3500659-21head08:05:263
347987199391,7cyclictest3482442-21/usr/sbin/munin07:15:153
347987199391,6cyclictest3581046-21sort12:00:173
3479871993910,24cyclictest3492521-21latency_hist07:45:013
3479862993928,7cyclictest0-21swapper/109:10:281
3479858993929,6cyclictest121ktimers/008:45:150
347985899392,11cyclictest3540123-21meminfo10:00:200
3479858993921,13cyclictest0-21swapper/012:10:160
3479858993919,16cyclictest13850irq/36-mmc110:20:000
347985899391,8cyclictest3519659-21/usr/sbin/munin09:00:230
347985899391,4cyclictest3558762-21irqstats10:55:170
347985899391,10cyclictest3500050-21memory08:05:200
347987199387,3cyclictest3547565-21awk10:20:253
347987199382,6cyclictest3501801-21taskset08:10:203
3479871993813,22cyclictest3559950-21latency_hist11:00:013
347987199381,10cyclictest3494265-21munin-node07:50:133
3479867993831,4cyclictest3493575-21unixbench_multi07:45:262
3479867993826,9cyclictest0-21swapper/207:20:002
347986799381,7cyclictest0-21swapper/210:55:102
347985899389,21cyclictest2376038-1kworker/0:2H+mmc_complete08:35:010
3479858993830,4cyclictest0-21swapper/007:50:010
347985899381,9cyclictest3573121-21unixbench_singl11:35:240
347985899381,9cyclictest3573121-21unixbench_singl11:35:240
347985899381,9cyclictest3556999-21/usr/sbin/munin10:50:180
347985899381,7cyclictest3529725-21sed09:30:200
347985899381,7cyclictest3495965-21/usr/sbin/munin07:55:100
347985899381,7cyclictest3485158-21sshd07:20:540
347985899381,7cyclictest3480584-21/usr/sbin/munin07:10:220
347985899381,6cyclictest3577167-21cron11:50:000
347985899381,6cyclictest3538398-21/usr/sbin/munin09:55:210
347985899381,11cyclictest3589958-21/usr/sbin/munin12:25:240
347985899381,10cyclictest3505245-21df08:20:150
347987199379,24cyclictest3585877-21latency_hist12:15:013
347987199372,6cyclictest3508296-21cron08:30:003
347987199371,9cyclictest3579322-21fschecks_time11:55:143
347987199371,9cyclictest3576044-21/usr/sbin/munin11:45:193
347987199371,9cyclictest3517942-21/usr/sbin/munin08:55:223
347987199371,9cyclictest3509805-21grep08:30:243
347987199371,8cyclictest3552310-21mailstats10:35:233
347987199371,8cyclictest3543371-21/usr/sbin/munin10:10:203
347987199371,7cyclictest3569699-21/usr/sbin/munin11:25:273
347987199371,6cyclictest3514158-21if_eth008:45:193
3479871993711,23cyclictest3594307-21latency_hist12:40:003
3479871993711,23cyclictest3594307-21latency_hist12:40:003
347987199370,11cyclictest3567829-21sed11:20:233
347986299378,6cyclictest131rcu_preempt11:55:121
3479862993731,4cyclictest3540879-21kworker/u8:1+events_unbound10:06:401
3479862993731,4cyclictest3540879-21kworker/u8:1+events_unbound10:06:401
3479862993726,8cyclictest3584001-21idleruntime-cro12:10:001
3479862993718,6cyclictest131rcu_preempt12:00:131
3479862993714,7cyclictest24-21ksoftirqd/107:40:261
347985899374,5cyclictest3524217-21/usr/sbin/munin09:15:160
3479858993728,6cyclictest3565407-21fschecks_count11:15:160
347985899371,9cyclictest3554372-21unixbench_singl10:40:270
347985899371,9cyclictest3522503-21/usr/sbin/munin09:10:160
347985899371,9cyclictest3487566-21/usr/sbin/munin07:30:150
347985899371,9cyclictest3483257-21cat07:15:250
347985899371,7cyclictest3586282-21cut12:15:180
347985899371,7cyclictest3552283-21/usr/sbin/munin10:35:220
347985899371,7cyclictest3541482-21expr10:05:160
347985899371,7cyclictest3541482-21expr10:05:150
347985899371,7cyclictest3486469-21/usr/sbin/munin07:25:250
347985899371,6cyclictest3593201-21/usr/sbin/munin12:35:180
347985899371,6cyclictest3593201-21/usr/sbin/munin12:35:180
347985899371,5cyclictest3570915-21/usr/sbin/munin11:30:200
347985899371,5cyclictest3570915-21/usr/sbin/munin11:30:200
347985899371,5cyclictest3550044-21sort10:30:180
347985899371,5cyclictest3543080-21/usr/sbin/munin10:10:140
347985899370,4cyclictest3516241-21sshd08:50:250
347987199368,25cyclictest0-21swapper/310:05:183
347987199368,25cyclictest0-21swapper/310:05:183
347987199362,11cyclictest3488909-21cron07:35:003
347987199361,9cyclictest3537515-21apt-get09:55:083
347987199361,7cyclictest3563759-21/usr/sbin/munin11:10:173
347987199361,7cyclictest3562149-21/usr/sbin/munin11:05:173
347987199361,7cyclictest3525802-21taskset09:20:023
347987199361,7cyclictest3525802-21taskset09:20:013
347987199361,6cyclictest3529763-21/usr/sbin/munin09:30:213
347987199361,6cyclictest3498082-21fschecks_time08:00:173
347987199361,6cyclictest3485150-21taskset07:20:473
347987199361,6cyclictest1-21systemd09:35:323
3479871993610,24cyclictest3505138-21/usr/sbin/munin08:20:223
347987199360,7cyclictest3582533-21apt-get12:05:133
3479862993629,5cyclictest3563150-21kworker/u8:2+events_unbound11:20:571
347985899362,4cyclictest3584162-21sed12:10:010
347985899362,3cyclictest3578028-21/usr/sbin/munin11:50:220
347985899361,9cyclictest3579717-21/usr/sbin/munin11:55:220
347985899361,9cyclictest3564486-21/usr/sbin/munin11:10:250
347985899361,9cyclictest3507112-21/usr/sbin/munin08:25:180
347985899361,8cyclictest3546552-21/usr/sbin/munin10:20:140
347985899361,8cyclictest3521529-21/usr/sbin/munin09:05:230
347985899361,7cyclictest3548438-21/usr/sbin/munin10:25:160
347985899361,6cyclictest3534490-21fschecks_count09:45:140
347985899361,11cyclictest3490981-21df_inode07:40:160
347985899361,10cyclictest3574696-21sed11:40:250
347985899361,10cyclictest3574696-21sed11:40:250
347985899360,7cyclictest3568677-21apt-get11:25:130
347987199359,23cyclictest3520070-21/usr/sbin/munin09:00:253
347987199356,26cyclictest0-21swapper/309:25:253
347987199352,6cyclictest3525558-21cron09:20:013
347987199352,4cyclictest3586538-21/usr/sbin/munin12:15:223
347987199352,4cyclictest3573790-21/usr/sbin/munin11:40:143
347987199352,4cyclictest3573790-21/usr/sbin/munin11:40:133
347987199351,9cyclictest3549959-21/usr/sbin/munin10:30:163
347987199351,5cyclictest3565386-21cut11:15:153
347987199351,5cyclictest3558381-21cut10:55:023
347987199351,5cyclictest3510853-21/usr/sbin/munin08:35:193
347987199351,4cyclictest3589349-21/usr/sbin/munin12:25:263
347987199351,4cyclictest3578168-21/usr/sbin/munin11:50:243
347987199351,4cyclictest3540270-21/usr/sbin/munin10:00:233
347987199351,4cyclictest3512267-21/usr/sbin/munin08:40:153
347987199351,10cyclictest629-21snmpd08:52:503
347987199350,7cyclictest3534353-21grep09:45:133
3479862993528,5cyclictest3549448-21kworker/u8:3+events_unbound10:46:411
3479862993526,5cyclictest0-21swapper/111:50:161
347985899351,5cyclictest3581464-21/usr/sbin/munin12:00:250
347985899351,3cyclictest3517270-21sshd08:55:120
347985899351,3cyclictest3512699-21/usr/sbin/munin08:40:230
347985899351,11cyclictest3567322-21uname11:20:170
3479871993425,4cyclictest0-21swapper/310:25:163
347987199342,4cyclictest3485748-21apt-get07:25:143
347987199341,8cyclictest3479999-21sed07:10:023
347987199341,5cyclictest3560592-21meminfo11:00:193
347986299348,5cyclictest131rcu_preempt08:10:131
3479862993425,4cyclictest0-21swapper/108:55:151
3479862993424,7cyclictest0-21swapper/108:30:151
3479862993423,7cyclictest3535803-21cron09:50:001
347985899341,9cyclictest3489385-21/usr/sbin/munin07:35:160
347985899341,9cyclictest3489385-21/usr/sbin/munin07:35:160
347985899341,5cyclictest3527967-21meminfo09:25:190
3479867993326,4cyclictest0-21swapper/212:20:252
3479867993325,4cyclictest0-21swapper/209:25:172
3479867993323,7cyclictest311ktimers/207:25:132
3479867993226,3cyclictest0-21swapper/212:10:162
3479867993224,4cyclictest0-21swapper/211:45:272
3479862993224,4cyclictest0-21swapper/109:00:161
3479862993222,5cyclictest3585709-21idleruntime-cro12:15:001
347986299321,7cyclictest24-21ksoftirqd/110:50:131
3479867993124,4cyclictest0-21swapper/210:25:242
3479867993122,6cyclictest208-21systemd-logind08:30:202
3479862993122,6cyclictest53-21kcompactd011:45:001
3479862993122,6cyclictest53-21kcompactd011:45:001
3479862993121,7cyclictest3593756-21grep12:35:251
3479862993121,7cyclictest3593756-21grep12:35:241
3479862993121,7cyclictest0-21swapper/108:00:271
3479862993117,5cyclictest131rcu_preempt08:55:001
347986299311,12cyclictest24-21ksoftirqd/107:30:121
347986299310,17cyclictest0-21swapper/108:20:121
3479867993024,3cyclictest3561609-21cron11:05:012
3479867993023,4cyclictest0-21swapper/210:05:272
3479867993023,4cyclictest0-21swapper/210:05:262
347986299309,13cyclictest24-21ksoftirqd/107:35:261
347986299309,13cyclictest24-21ksoftirqd/107:35:251
347986299304,3cyclictest131rcu_preempt11:15:121
3479862993021,6cyclictest231ktimers/112:20:131
3479862993017,5cyclictest131rcu_preempt10:20:161
3479862993010,12cyclictest24-21ksoftirqd/108:20:001
3479867992922,5cyclictest3477837-21kworker/u8:0+events_unbound07:11:412
3479867992922,4cyclictest3573540-21sh11:40:002
3479867992922,4cyclictest3573540-21sh11:39:592
3479867992922,4cyclictest3497475-21latency_hist08:00:022
3479867992922,4cyclictest0-21swapper/208:00:182
3479867992921,6cyclictest3533739-21kworker/u8:0+events_unbound09:45:362
3479867992921,4cyclictest0-21swapper/209:10:272
3479867992918,6cyclictest0-21swapper/211:35:002
3479867992918,6cyclictest0-21swapper/211:35:002
347986299295,7cyclictest24-21ksoftirqd/109:30:261
3479862992921,4cyclictest0-21swapper/111:00:001
3479862992920,4cyclictest0-21swapper/108:40:011
3479862992918,7cyclictest0-21swapper/110:40:001
3479862992915,9cyclictest0-21swapper/111:25:161
3479862992914,4cyclictest3575068-21kworker/u8:1-events_freezable_power_11:45:241
3479862992912,3cyclictest131rcu_preempt10:25:241
3479867992821,4cyclictest0-21swapper/207:40:012
3479867992821,4cyclictest0-21swapper/207:40:002
3479867992820,4cyclictest0-21swapper/210:40:282
347986299288,11cyclictest131rcu_preempt12:15:161
347986299284,6cyclictest24-21ksoftirqd/107:10:271
347986299284,3cyclictest131rcu_preempt07:55:121
3479862992822,4cyclictest3482320-21kworker/u8:3+events_unbound07:18:411
3479862992819,6cyclictest0-21swapper/111:10:011
3479862992812,6cyclictest24-21ksoftirqd/111:35:241
3479862992812,6cyclictest24-21ksoftirqd/111:35:241
3479867992722,3cyclictest3537594-21apt-config09:55:022
3479867992721,3cyclictest0-21swapper/212:30:002
3479867992721,3cyclictest0-21swapper/211:45:002
3479867992721,3cyclictest0-21swapper/211:45:002
3479867992720,4cyclictest0-21swapper/210:30:272
3479867992720,4cyclictest0-21swapper/209:35:252
3479867992719,4cyclictest0-21swapper/209:20:252
3479867992719,4cyclictest0-21swapper/209:20:242
347986299278,3cyclictest131rcu_preempt07:55:021
347986299277,13cyclictest24-21ksoftirqd/109:15:231
347986299277,12cyclictest24-21ksoftirqd/108:25:201
347986299277,11cyclictest24-21ksoftirqd/109:55:011
3479862992719,4cyclictest0-21swapper/110:00:011
3479862992719,4cyclictest0-21swapper/108:40:181
3479862992718,6cyclictest3544078-21unixbench_multi10:10:281
3479862992716,4cyclictest3544400-21kworker/u8:2+events_unbound10:15:421
3479862992715,8cyclictest231ktimers/112:25:131
3479862992715,5cyclictest131rcu_preempt11:00:211
3479862992715,5cyclictest131rcu_preempt10:40:171
3479867992618,5cyclictest3563305-21cron11:10:012
3479867992618,4cyclictest0-21swapper/208:45:172
3479867992611,7cyclictest32-21ksoftirqd/211:15:242
347986299269,3cyclictest0-21swapper/109:27:571
347986299265,11cyclictest0-21swapper/110:04:261
3479862992620,4cyclictest3533739-21kworker/u8:0+events_unbound09:44:411
3479862992613,3cyclictest131rcu_preempt11:10:261
347986299260,6cyclictest24-21ksoftirqd/112:34:591
347986299260,6cyclictest24-21ksoftirqd/112:34:591
3479867992519,4cyclictest3542630-21kworker/u8:0+events_unbound10:20:412
3479867992519,4cyclictest3528720-21kworker/u8:2+events_unbound09:30:322
3479867992517,4cyclictest3567968-21grep11:20:232
3479867992511,11cyclictest32-21ksoftirqd/209:05:212
347986299253,7cyclictest24-21ksoftirqd/109:05:241
3479862992516,7cyclictest3484350-21kworker/u8:0+events_unbound07:29:501
3479867992418,4cyclictest3499217-21kworker/u8:0+events_unbound08:44:012
3479867992417,4cyclictest3545782-21tail10:15:252
3479867992415,6cyclictest0-21swapper/208:20:002
347986799241,12cyclictest0-21swapper/212:05:232
347986299247,4cyclictest131rcu_preempt08:45:221
347986299245,12cyclictest24-21ksoftirqd/108:10:021
347986299245,10cyclictest24-21ksoftirqd/107:45:191
3479862992416,6cyclictest3528720-21kworker/u8:2+events_unbound09:38:121
3479867992317,4cyclictest35641452sleep211:10:242
3479867992317,4cyclictest3533739-21kworker/u8:0-ext4-rsv-conversion09:45:012
3479867992317,3cyclictest0-21swapper/209:00:212
3479867992314,6cyclictest0-21swapper/210:35:162
347986799226,8cyclictest131rcu_preempt08:10:112
347986799226,7cyclictest131rcu_preempt08:20:162
3479867992216,4cyclictest3533739-21kworker/u8:0+events_unbound09:57:212
3479867992216,3cyclictest3556479-21cron10:50:002
3479867992216,3cyclictest0-21swapper/212:00:262
3479867992216,3cyclictest0-21swapper/211:55:142
3479867992215,5cyclictest3543482-21latency10:10:232
3479867992215,5cyclictest3488443-21unixbench_multi07:30:262
3479867992215,4cyclictest0-21swapper/208:05:262
3479867992116,3cyclictest0-21swapper/212:15:252
3479867992115,4cyclictest3499217-21kworker/u8:0+events_unbound08:25:282
3479867992115,3cyclictest0-21swapper/208:55:262
3479867992112,7cyclictest3540590-21wc10:00:252
3479867992111,6cyclictest3511919-21cron08:40:002
3479867992012,5cyclictest0-21swapper/211:50:232
3479867992011,6cyclictest0-21swapper/211:25:222
347986799199,7cyclictest3516161-21open_inodes08:50:232
3479867991911,5cyclictest3484187-21fschecks_count07:20:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional