You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-24 - 20:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot5s.osadl.org (updated Fri Jul 24, 2026 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2514175994940,6cyclictest2615955-21grep12:00:271
2514171994934,11cyclictest121ktimers/008:55:140
2514184994839,7cyclictest2625090-21kworker/u8:0+events_unbound12:35:213
2514184994620,22cyclictest0-21swapper/308:55:273
2514179994634,4cyclictest131rcu_preempt10:45:002
2514171994416,10cyclictest2610881-21sed11:45:260
2514171994416,10cyclictest2610881-21sed11:45:260
2514179994337,4cyclictest2526536-21kworker/u8:0+events_unbound07:59:222
251418499421,9cyclictest2614925-21munin-node12:00:133
251418499421,9cyclictest2611509-21/usr/sbin/munin11:50:263
251418499421,9cyclictest2557670-21sshd09:11:273
251418499421,9cyclictest2520187-21/usr/sbin/munin07:25:163
251418499421,8cyclictest53-21kcompactd008:24:443
251418499421,11cyclictest2606517-21/usr/sbin/munin11:35:143
251418499421,11cyclictest2606517-21/usr/sbin/munin11:35:143
2514171994214,10cyclictest2586969-21latency_hist10:40:010
2514171994214,10cyclictest2568380-21sed09:45:010
2514171994210,15cyclictest2514313-21date07:10:010
2514184994132,6cyclictest2533021-21grep08:00:253
251418499411,9cyclictest2564354-21unixbench_singl09:30:253
251418499411,8cyclictest2625816-21meminfo12:30:203
251418499411,7cyclictest2580993-21/usr/sbin/munin10:20:233
251418499411,6cyclictest2587791-21ntp_states10:40:213
2514171994124,8cyclictest121ktimers/008:45:130
251418499408,9cyclictest2553894-21cat09:00:243
251418499402,8cyclictest2616947-21/usr/sbin/munin12:05:153
251418499402,8cyclictest2616947-21/usr/sbin/munin12:05:153
251418499401,9cyclictest2613557-21/usr/sbin/munin11:55:193
251418499401,9cyclictest2605686-21cron11:32:003
251418499401,9cyclictest2578977-21/usr/sbin/munin10:15:163
251418499401,9cyclictest2555432-21/usr/sbin/munin09:05:233
251418499401,8cyclictest2558753-21/usr/sbin/munin09:15:213
251418499401,7cyclictest2545191-21/usr/sbin/munin08:35:273
251418499401,7cyclictest2530285-21/usr/sbin/munin07:55:143
2514184994013,24cyclictest438-21cron10:05:013
251418499401,11cyclictest2535523-21diskstats08:10:173
251418499401,10cyclictest2516974-21/usr/sbin/munin07:15:203
2514179994029,8cyclictest0-21swapper/211:20:002
2514179994014,5cyclictest0-21swapper/207:15:132
2514175994029,8cyclictest2614680-21cron12:00:001
2514171994030,6cyclictest0-21swapper/010:55:270
2514171994017,3cyclictest2545432-21users08:35:280
251418499393,9cyclictest2514446-21df_inode07:10:153
2514184993926,8cyclictest2538574-21apt08:20:013
251418499391,8cyclictest2566102-21/usr/sbin/munin09:35:263
251418499391,7cyclictest2610091-21/usr/sbin/munin11:45:173
251418499391,7cyclictest2610091-21/usr/sbin/munin11:45:173
251418499391,6cyclictest2594372-21diskstats11:00:153
251418499391,6cyclictest2591102-21/usr/sbin/munin10:50:213
251418499391,11cyclictest2596399-21meminfo11:05:173
251418499391,10cyclictest2543093-21/usr/sbin/munin08:30:183
2514179993927,7cyclictest0-21swapper/210:10:252
2514179993920,7cyclictest131rcu_preempt08:40:122
2514175993931,5cyclictest2566513-21sendmail09:40:001
2514171993913,9cyclictest2564165-21switchtime09:30:240
251418499387,7cyclictest2567236-21wc09:40:213
251418499383,6cyclictest2572422-21ntp_states09:55:263
251418499383,6cyclictest2572422-21ntp_states09:55:253
251418499382,4cyclictest2559930-21/usr/sbin/munin09:20:253
251418499381,9cyclictest2608556-21/usr/sbin/munin11:40:193
251418499381,9cyclictest2608556-21/usr/sbin/munin11:40:183
251418499381,9cyclictest2569026-21ntp_states09:45:213
251418499381,9cyclictest2548220-21memory08:45:193
251418499381,9cyclictest2546790-21/usr/sbin/munin08:40:243
251418499381,9cyclictest2524035-21ntp_states07:35:263
251418499381,8cyclictest2597709-21/usr/sbin/munin11:10:133
251418499381,7cyclictest2578055-21sshd10:10:283
251418499381,7cyclictest2522302-21/usr/sbin/munin07:30:253
251418499381,7cyclictest2522302-21/usr/sbin/munin07:30:243
251418499381,6cyclictest2584785-21unixbench_multi10:30:253
251418499381,11cyclictest2599472-21df_inode11:15:163
251418499381,11cyclictest2589258-21irqstats10:45:173
251418499381,11cyclictest2534553-21wc08:05:243
251418499381,10cyclictest2586484-21tail10:35:233
251418499381,10cyclictest2519407-21grep07:20:283
2514175993829,5cyclictest2538578-21latency_hist08:20:011
2514175993829,5cyclictest0-21swapper/111:20:031
2514175993828,7cyclictest2543659-21grep08:30:251
251417199388,12cyclictest2627231-21fschecks_count12:35:150
2514171993825,9cyclictest2516642-21/usr/sbin/munin07:15:150
2514171993824,11cyclictest2535405-21apt-get08:10:130
2514171993815,3cyclictest2579751-21users10:15:250
2514171993813,20cyclictest2538839-21latency_hist08:20:010
2514171993811,7cyclictest2613216-21apt-get11:55:140
251418499376,2cyclictest0-21swapper/310:07:563
251418499373,7cyclictest2583346-21sshd10:27:273
251418499371,9cyclictest2601188-21df11:20:153
251418499371,9cyclictest2592430-21df10:55:153
251418499371,9cyclictest2527329-21memory07:45:203
251418499371,6cyclictest2603405-21ps11:25:233
251418499371,5cyclictest2620000-21cron12:14:593
251418499371,5cyclictest2620000-21cron12:14:593
251418499371,10cyclictest2561737-21df09:25:133
2514171993727,7cyclictest2520664-21/usr/sbin/munin07:25:250
2514171993726,7cyclictest2529215-21/usr/sbin/munin07:50:250
2514171993726,7cyclictest2529215-21/usr/sbin/munin07:50:240
2514171993724,10cyclictest2533836-21/usr/sbin/munin08:05:160
2514171993720,4cyclictest2542753-21df08:30:140
2514171993714,20cyclictest0-21swapper/011:40:250
2514171993714,20cyclictest0-21swapper/011:40:240
251418499369,4cyclictest2540928-21fschecks_time08:25:173
251418499363,5cyclictest2624119-21memory12:25:203
251418499361,9cyclictest2529278-21/usr/sbin/munin07:50:263
251418499361,9cyclictest2529278-21/usr/sbin/munin07:50:253
251418499361,8cyclictest2570718-21ntp_states09:50:213
251418499361,8cyclictest2525626-21meminfo07:40:193
2514184993616,16cyclictest0-21swapper/312:15:173
2514179993618,6cyclictest131rcu_preempt11:25:132
2514179993618,6cyclictest131rcu_preempt09:20:122
2514179993615,6cyclictest131rcu_preempt08:10:132
2514175993626,7cyclictest0-21swapper/109:10:011
2514171993623,10cyclictest2601421-21/usr/sbin/munin11:20:180
2514171993622,11cyclictest2581120-21/usr/sbin/munin10:20:250
2514171993622,10cyclictest2622505-21/usr/sbin/munin12:20:210
2514171993620,13cyclictest2587664-21/usr/sbin/munin10:40:180
2514171993619,13cyclictest2561730-21cstates09:25:130
2514171993616,6cyclictest2546518-21memory08:40:190
2514171993616,15cyclictest2582722-21ps10:25:220
2514171993612,8cyclictest2530840-21ntpq07:55:240
251418499351,8cyclictest2550043-21/usr/sbin/munin08:50:223
251418499351,4cyclictest2622128-21cut12:20:173
2514179993517,6cyclictest131rcu_preempt10:35:112
2514179993510,10cyclictest0-21swapper/212:38:282
2514175993526,5cyclictest0-21swapper/107:15:191
2514175993526,4cyclictest0-21swapper/108:55:181
2514175993523,8cyclictest2628540-21cron12:40:011
2514175993521,9cyclictest231ktimers/107:55:151
2514171993523,9cyclictest2541188-21/usr/sbin/munin08:25:200
2514171993523,7cyclictest2525759-21awk07:40:230
2514171993523,7cyclictest2518478-21/usr/sbin/munin07:20:170
2514171993522,8cyclictest2571962-21df09:55:180
2514171993522,8cyclictest2571962-21df09:55:180
2514171993522,10cyclictest2624448-21/usr/sbin/munin12:25:240
2514171993522,10cyclictest2598329-21/usr/sbin/munin11:10:240
2514171993521,10cyclictest2521914-21fschecks_count07:30:170
2514171993521,10cyclictest2521914-21fschecks_count07:30:160
2514171993520,11cyclictest2585259-21cron10:35:000
2514171993517,13cyclictest2612529-21grep11:50:260
2514171993513,19cyclictest2620499-21cat12:15:160
2514171993511,21cyclictest2616393-21apt-get12:05:020
2514171993511,21cyclictest2616393-21apt-get12:05:020
2514171993511,21cyclictest2599976-21proc_pri11:15:240
2514179993428,4cyclictest2574999-21kworker/u8:1+events_unbound10:09:512
2514179993424,6cyclictest0-21swapper/211:10:012
2514179993415,5cyclictest131rcu_preempt11:55:132
2514179993410,14cyclictest32-21ksoftirqd/211:30:242
2514175993424,7cyclictest2564382-21grep09:30:241
2514175993424,5cyclictest0-21swapper/109:20:001
2514171993427,4cyclictest2550948-21munin-plugin-st08:55:000
2514171993427,3cyclictest2602781-21apt-get11:25:130
2514171993424,7cyclictest2604576-21/usr/sbin/munin11:30:130
2514171993424,7cyclictest2526989-21/usr/sbin/munin07:45:150
2514171993422,10cyclictest2539566-21/usr/sbin/munin08:20:200
2514171993421,8cyclictest1-21systemd10:04:460
2514171993420,9cyclictest2525015-21latency_hist07:40:010
2514171993419,12cyclictest2625486-21/usr/sbin/munin12:30:140
2514171993418,3cyclictest2607468-21users11:35:250
2514171993418,3cyclictest2607468-21users11:35:250
2514171993417,12cyclictest2568641-21fschecks_time09:45:150
2514171993416,3cyclictest2596062-21df11:05:130
2514171993412,19cyclictest2594626-21kernelversion11:00:180
2514179993327,4cyclictest2597034-21kworker/u8:2+events_unbound11:40:092
2514179993327,4cyclictest2597034-21kworker/u8:2+events_unbound11:40:082
2514179993327,4cyclictest2519568-21kworker/u8:1+events_unbound07:29:162
2514179993326,4cyclictest0-21swapper/209:40:272
2514179993312,7cyclictest32-21ksoftirqd/208:20:022
2514179993312,11cyclictest0-21swapper/212:26:562
251417999331,18cyclictest0-21swapper/211:50:122
2514175993323,5cyclictest0-21swapper/107:20:291
2514171993323,7cyclictest2591319-21/usr/sbin/munin10:50:240
2514171993323,7cyclictest2559933-21apt-get09:20:120
2514171993323,7cyclictest2556691-21/usr/sbin/munin09:10:160
2514171993323,7cyclictest2532560-21/usr/sbin/munin08:00:210
2514171993321,9cyclictest2618542-21/usr/sbin/munin12:10:140
2514171993321,9cyclictest2618542-21/usr/sbin/munin12:10:130
2514171993321,9cyclictest2615487-21/usr/sbin/munin12:00:220
2514171993319,11cyclictest2570151-21munin-node09:50:110
2514171993315,14cyclictest2575368-21ls10:05:130
2514171993315,13cyclictest2555103-21if_eth009:05:170
2514171993312,16cyclictest2589185-21if_err_eth010:45:160
251417999326,6cyclictest32-21ksoftirqd/208:00:162
2514179993225,4cyclictest0-21swapper/211:00:162
2514179993225,3cyclictest0-21swapper/208:55:272
2514179993224,4cyclictest0-21swapper/212:10:012
2514179993224,4cyclictest0-21swapper/212:10:012
2514179993220,5cyclictest2618160-21kworker/u8:1+events_unbound12:17:122
2514175993225,4cyclictest0-21swapper/110:40:011
2514175993224,4cyclictest0-21swapper/112:25:251
2514175993221,7cyclictest2579731-21grep10:15:261
2514175993220,9cyclictest2569917-21cron09:50:001
2514171993226,3cyclictest2565034-21apt-get09:35:130
2514171993222,7cyclictest2577729-21/usr/sbin/munin10:10:230
251417999319,13cyclictest32-21ksoftirqd/209:00:182
2514179993124,5cyclictest2526536-21kworker/u8:0+events_unbound08:06:402
2514179993124,4cyclictest2515798-21cron07:15:012
2514179993121,7cyclictest2519819-21sh07:25:002
2514179993119,8cyclictest311ktimers/211:45:142
2514179993119,8cyclictest311ktimers/211:45:132
2514179993114,6cyclictest131rcu_preempt08:20:162
2514179993114,3cyclictest131rcu_preempt07:40:242
2514179993110,11cyclictest32-21ksoftirqd/209:55:302
2514179993110,11cyclictest32-21ksoftirqd/209:55:302
2514175993122,6cyclictest0-21swapper/109:50:161
2514171993121,7cyclictest2553658-21/usr/sbin/munin09:00:210
251417999309,13cyclictest32-21ksoftirqd/209:30:232
251417999308,12cyclictest32-21ksoftirqd/207:35:182
251417999305,8cyclictest32-21ksoftirqd/211:35:202
251417999305,8cyclictest32-21ksoftirqd/211:35:202
2514179993023,5cyclictest2618160-21kworker/u8:1+events_unbound12:20:412
2514179993023,4cyclictest2557990-21idleruntime-cro09:15:012
2514179993023,4cyclictest2547984-21fschecks_time08:45:172
2514179993015,9cyclictest131rcu_preempt10:50:242
2514179993013,6cyclictest131rcu_preempt08:50:122
2514179993011,5cyclictest131rcu_preempt12:00:272
2514179993011,11cyclictest32-21ksoftirqd/207:30:292
2514179993011,11cyclictest32-21ksoftirqd/207:30:282
2514179993010,4cyclictest0-21swapper/208:35:002
2514179993010,3cyclictest131rcu_preempt10:22:492
2514179993010,12cyclictest131rcu_preempt07:45:252
2514175993022,4cyclictest0-21swapper/109:25:251
2514175993020,6cyclictest0-21swapper/110:30:251
2514175993019,7cyclictest0-21swapper/108:55:011
2514179992923,4cyclictest2576730-21kworker/u8:2+events_unbound10:15:412
251417999292,13cyclictest0-21swapper/209:09:452
2514179992916,3cyclictest131rcu_preempt12:13:132
2514179992916,3cyclictest131rcu_preempt12:13:132
2514179992915,4cyclictest131rcu_preempt09:50:172
251417999290,7cyclictest2530044-21cron07:55:012
251417999290,7cyclictest2530044-21cron07:55:002
2514175992922,4cyclictest0-21swapper/112:10:281
2514175992922,4cyclictest0-21swapper/112:10:271
2514175992921,5cyclictest0-21swapper/111:15:261
2514175992921,5cyclictest0-21swapper/107:35:011
2514175992921,5cyclictest0-21swapper/107:35:011
2514175992920,5cyclictest0-21swapper/111:05:011
251417999285,12cyclictest32-21ksoftirqd/210:55:182
2514179992821,4cyclictest0-21swapper/208:25:262
2514179992817,3cyclictest131rcu_preempt10:05:002
2514179992816,8cyclictest1-21systemd09:29:392
2514179992810,3cyclictest131rcu_preempt11:24:562
2514175992820,5cyclictest145-21jbd2/mmcblk1p2-811:30:051
2514175992820,4cyclictest0-21swapper/108:25:171
2514175992818,7cyclictest2593385-21grep10:55:271
2514175992818,7cyclictest2583571-21idleruntime-cro10:30:001
251417999273,11cyclictest2588373-21kworker/u8:2-events_freezable_power_10:48:042
2514179992710,5cyclictest131rcu_preempt09:15:262
251417999266,12cyclictest32-21ksoftirqd/208:35:252
251417999266,12cyclictest131rcu_preempt09:45:142
2514179992618,4cyclictest0-21swapper/211:05:242
2514175992620,3cyclictest0-21swapper/109:00:221
2514175992619,4cyclictest0-21swapper/108:05:251
2514175992614,8cyclictest1-21systemd08:49:311
2514175992518,4cyclictest2572849-21grep09:55:301
2514175992518,4cyclictest2572849-21grep09:55:301
2514175992518,4cyclictest2529604-21sed07:50:271
2514175992518,4cyclictest2529604-21sed07:50:271
2514175992518,4cyclictest2515802-21kthreadd07:15:001
2514175992518,4cyclictest0-21swapper/107:35:301
2514175992516,6cyclictest2560094-21cut09:20:171
2514175992516,6cyclictest2546223-21tune2fs08:40:161
2514175992516,6cyclictest231ktimers/108:35:121
2514175992515,6cyclictest0-21swapper/107:30:011
2514175992418,3cyclictest0-21swapper/110:40:211
2514175992418,3cyclictest0-21swapper/110:10:211
2514175992418,3cyclictest0-21swapper/108:10:261
2514175992418,3cyclictest0-21swapper/108:00:241
2514175992415,6cyclictest2591476-21sort10:50:251
2514175992415,5cyclictest0-21swapper/112:30:271
2514175992415,5cyclictest0-21swapper/111:05:151
2514175992414,7cyclictest2566983-21tune2fs09:40:171
2514175992412,9cyclictest2581417-21grep10:20:251
251417599232,16cyclictest164-21systemd-journal12:15:161
2514175992317,3cyclictest0-21swapper/111:10:261
2514175992317,3cyclictest0-21swapper/108:20:131
2514175992313,7cyclictest2608804-21expr11:40:231
2514175992313,7cyclictest2608804-21expr11:40:231
2514175992214,6cyclictest231ktimers/112:05:041
2514175992214,6cyclictest231ktimers/112:05:041
2514175992213,6cyclictest0-21swapper/111:45:271
2514175992213,6cyclictest0-21swapper/111:45:261
2514175992115,3cyclictest0-21swapper/112:25:001
2514175992115,3cyclictest0-21swapper/110:05:221
2514175992114,4cyclictest2556662-21fschecks_count09:10:161
251417599201,4cyclictest0-21swapper/107:45:201
2514175992013,4cyclictest0-21swapper/110:00:161
251417599201,14cyclictest2589136-21sort10:45:151
251417599191,8cyclictest2604248-21grep11:30:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional