You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 18:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Sat Nov 29, 2025 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2350027995140,7cyclictest2420430-21grep10:30:253
2350027995038,8cyclictest2443079-21grep11:40:003
2350024994739,5cyclictest2359065-21cron07:35:002
2350015994739,5cyclictest2422612-21munin-plugin-st10:40:000
2350019994535,6cyclictest2375680-21unixbench_multi08:20:251
2350015994511,16cyclictest0-21swapper/011:45:140
2350024994333,6cyclictest2403637-21sh09:45:012
2350015994221,15cyclictest2377828-21sh08:30:000
2350015994221,13cyclictest0-21swapper/011:30:130
2350015994221,13cyclictest0-21swapper/008:10:140
2350024994133,4cyclictest2423261-21seq10:40:202
2350024994131,6cyclictest0-21swapper/207:20:282
2350019994133,4cyclictest0-21swapper/108:50:181
2350019994133,4cyclictest0-21swapper/108:50:171
2350015994111,10cyclictest2463359-21awk12:35:260
2350024994030,6cyclictest2446085-21unixbench_singl11:45:292
2350015994029,8cyclictest2388280-21sendmail09:00:000
2350015994015,9cyclictest2363613-21timerandwakeup07:45:250
2350024993932,5cyclictest2396502-21kworker/u8:0+events_unbound09:30:372
2350024993828,6cyclictest0-21swapper/211:40:282
2350015993832,3cyclictest2400456-21apt-get09:35:130
2350015993820,13cyclictest0-21swapper/010:55:150
2350015993815,18cyclictest2403811-21latency_hist09:45:010
2350015993811,16cyclictest0-21swapper/012:10:520
2350027993727,6cyclictest0-21swapper/312:20:003
2350024993728,6cyclictest2356908-21unixbench_multi07:25:262
2350024993727,6cyclictest0-21swapper/212:35:272
2350024993727,6cyclictest0-21swapper/207:20:012
2350024993726,7cyclictest0-21swapper/210:25:272
2350024993725,8cyclictest0-21swapper/210:00:262
2350015993730,4cyclictest2441817-21fschecks_time11:35:170
2350015993724,10cyclictest2425104-21/usr/sbin/munin10:45:230
2350015993723,10cyclictest2434778-21apt-get11:15:130
2350015993723,10cyclictest2407405-21/usr/sbin/munin09:55:180
2350015993721,3cyclictest2366690-21ntp_states07:55:220
2350015993719,12cyclictest2368057-21fschecks_time08:00:180
2350015993718,15cyclictest2415320-21tail10:15:270
2350027993627,6cyclictest2385260-21fschecks_count08:50:173
2350027993627,6cyclictest2385260-21fschecks_count08:50:173
2350027993625,8cyclictest2436268-21sendmail11:20:003
2350019993628,5cyclictest2410297-21kworker/u8:2+events_unbound10:45:441
2350019993626,6cyclictest2373074-21fschecks_count08:15:171
2350015993629,3cyclictest2455468-21apt-get12:15:140
2350015993624,8cyclictest2431375-21munin-node11:05:120
2350015993623,10cyclictest2405005-21cron09:46:000
2350015993622,9cyclictest0-21swapper/011:50:120
2350015993622,11cyclictest2352207-21df07:15:040
2350027993528,3cyclictest0-21swapper/308:10:203
2350027993527,5cyclictest2353959-21sendmail07:20:013
2350027993527,5cyclictest0-21swapper/309:00:003
2350027993527,4cyclictest0-21swapper/307:30:163
2350027993526,6cyclictest0-21swapper/312:40:003
2350027993526,4cyclictest0-21swapper/308:40:233
2350027993525,6cyclictest0-21swapper/311:25:193
2350024993527,4cyclictest0-21swapper/210:50:172
2350024993526,4cyclictest0-21swapper/212:10:012
2350024993524,7cyclictest0-21swapper/212:25:282
2350024993524,7cyclictest0-21swapper/207:50:002
2350019993527,4cyclictest0-21swapper/110:25:421
2350019993527,4cyclictest0-21swapper/107:20:181
2350019993524,7cyclictest0-21swapper/107:55:001
2350015993525,6cyclictest2449444-21grep11:55:270
2350015993524,8cyclictest2361142-21/usr/sbin/munin07:40:170
2350015993524,8cyclictest2361142-21/usr/sbin/munin07:40:160
2350015993524,7cyclictest2385176-21/usr/sbin/munin08:50:150
2350015993524,7cyclictest2385176-21/usr/sbin/munin08:50:150
2350015993523,9cyclictest2438731-21/usr/sbin/munin11:25:240
2350015993523,9cyclictest2365271-21/usr/sbin/munin07:50:280
2350015993523,10cyclictest2358086-21/usr/sbin/munin07:30:200
2350015993522,8cyclictest2406022-21meminfo09:50:200
2350015993522,10cyclictest2393735-21/usr/sbin/munin09:15:150
2350015993522,10cyclictest2390661-21/usr/sbin/munin09:05:210
2350015993521,11cyclictest2418304-21ntp_states10:25:230
2350015993521,10cyclictest2430116-21/usr/sbin/munin11:00:200
2350015993521,10cyclictest2408996-21/usr/sbin/munin10:00:120
2350015993519,12cyclictest2389522-21grep09:00:270
2350027993422,9cyclictest391ktimers/309:50:143
2350024993427,4cyclictest2360791-21sendmail07:40:002
2350024993425,5cyclictest0-21swapper/207:55:262
2350024993424,7cyclictest2455233-21auditctl12:15:002
2350024993423,7cyclictest0-21swapper/209:00:272
2350019993426,4cyclictest0-21swapper/108:00:191
2350015993428,3cyclictest2398750-21apt-get09:30:120
2350015993427,4cyclictest2395350-21apt-get09:20:120
2350015993424,7cyclictest2416881-21/usr/sbin/munin10:20:260
2350015993424,7cyclictest2411357-21/usr/sbin/munin10:05:210
2350015993423,8cyclictest2461317-21/usr/sbin/munin12:30:250
2350015993423,7cyclictest2419594-21/usr/sbin/munin10:30:160
2350015993422,9cyclictest2459961-21/usr/sbin/munin12:25:280
2350015993422,9cyclictest2433077-21/usr/sbin/munin11:10:130
2350015993422,9cyclictest2373629-21/usr/sbin/munin08:15:240
2350015993422,9cyclictest2356170-21/usr/sbin/munin07:25:180
2350015993422,8cyclictest2426362-21df10:50:150
2350015993422,7cyclictest2378395-21irqstats08:30:170
2350015993421,10cyclictest2350505-21/usr/sbin/munin07:10:180
2350015993420,10cyclictest0-21swapper/011:40:160
2350015993418,9cyclictest0-21swapper/012:20:150
2350015993416,13cyclictest2453321-21apt-get12:10:000
2350015993416,10cyclictest0-21swapper/008:20:120
2350015993415,16cyclictest2437744-21sshd11:22:200
2350015993413,18cyclictest2451289-21taskset12:01:020
2350027993326,4cyclictest0-21swapper/310:20:013
2350027993323,7cyclictest2453993-21tune2fs12:10:173
2350027993323,5cyclictest0-21swapper/308:35:273
2350027993322,6cyclictest0-21swapper/309:25:253
2350027993310,4cyclictest40-21ksoftirqd/308:15:253
2350024993327,4cyclictest2422611-21cron10:40:002
2350024993324,5cyclictest0-21swapper/212:00:002
2350024993323,6cyclictest0-21swapper/212:20:272
2350024993323,6cyclictest0-21swapper/211:05:262
2350019993325,4cyclictest0-21swapper/109:00:181
2350015993323,7cyclictest2392727-21/usr/sbin/munin09:10:240
2350015993323,7cyclictest2360149-21/usr/sbin/munin07:35:290
2350015993321,9cyclictest2370522-21/usr/sbin/munin08:05:280
2350015993321,8cyclictest2381808-21/usr/sbin/munin08:40:190
2350015993318,3cyclictest2354313-21df07:20:170
235002799329,6cyclictest40-21ksoftirqd/310:25:013
2350027993224,4cyclictest0-21swapper/308:45:243
2350027993223,4cyclictest0-21swapper/307:55:163
2350027993220,8cyclictest2375659-21grep08:20:253
2350024993222,6cyclictest0-21swapper/211:00:262
2350024993222,6cyclictest0-21swapper/208:30:272
2350024993221,7cyclictest0-21swapper/207:40:262
2350024993221,7cyclictest0-21swapper/207:40:262
2350024993221,7cyclictest0-21swapper/207:10:262
2350019993224,5cyclictest2412384-21munin-plugin-st10:10:001
2350019993221,6cyclictest0-21swapper/108:40:231
2350015993222,7cyclictest2422965-21df10:40:160
2350015993222,7cyclictest2412610-21munin-node10:10:130
2350015993222,7cyclictest2380444-21/usr/sbin/munin08:35:250
2350015993221,8cyclictest2383509-21df_inode08:45:160
235002799319,6cyclictest40-21ksoftirqd/311:40:233
2350027993123,4cyclictest0-21swapper/311:55:283
2350027993123,3cyclictest0-21swapper/312:00:183
2350024993124,4cyclictest0-21swapper/208:10:282
2350024993123,5cyclictest2410297-21kworker/u8:2+events_unbound10:20:432
2350024993123,4cyclictest0-21swapper/209:35:362
2350024993121,7cyclictest2365887-21audit_send_reply07:55:012
2350019993123,4cyclictest0-21swapper/109:25:161
2350019993112,5cyclictest131rcu_preempt10:55:141
2350027993023,4cyclictest0-21swapper/308:05:183
2350027993023,3cyclictest0-21swapper/307:40:263
2350027993023,3cyclictest0-21swapper/307:40:263
2350027993022,4cyclictest0-21swapper/312:25:283
2350027993022,4cyclictest0-21swapper/312:20:273
2350027993021,6cyclictest2437471-21timerwakeupswit11:20:263
2350027993021,4cyclictest0-21swapper/310:00:173
2350027993014,4cyclictest131rcu_preempt10:13:213
2350027993011,8cyclictest2350005-21cyclictest09:06:063
2350024993021,5cyclictest2443064-21cron11:40:002
2350019993023,5cyclictest2434321-21kworker/u8:0+events_unbound11:35:531
2350027992921,4cyclictest0-21swapper/310:05:183
2350027992920,6cyclictest391ktimers/310:35:143
2350027992920,6cyclictest2401242-21sort09:35:273
2350027992920,5cyclictest0-21swapper/309:25:013
2350027992916,10cyclictest2399204-21/usr/sbin/munin09:30:213
2350027992910,11cyclictest2350005-21cyclictest08:25:313
2350024992921,4cyclictest0-21swapper/208:55:002
2350024992921,4cyclictest0-21swapper/208:55:002
2350019992922,5cyclictest2394740-21kworker/u8:1+events_unbound09:35:361
2350019992922,4cyclictest0-21swapper/107:25:261
2350019992921,4cyclictest0-21swapper/111:05:261
2350019992919,6cyclictest0-21swapper/110:40:281
2350019992919,5cyclictest0-21swapper/109:55:281
235002799287,4cyclictest40-21ksoftirqd/309:15:253
2350027992821,4cyclictest0-21swapper/308:05:013
2350027992820,4cyclictest0-21swapper/311:00:183
2350027992820,4cyclictest0-21swapper/307:30:003
2350027992819,5cyclictest0-21swapper/309:05:003
2350024992821,4cyclictest0-21swapper/210:10:272
2350024992820,4cyclictest0-21swapper/211:35:002
2350024992820,4cyclictest0-21swapper/210:55:262
2350024992817,6cyclictest2373073-21fschecks_count08:15:172
2350024992814,9cyclictest1-21systemd11:10:472
2350019992818,8cyclictest24-21ksoftirqd/109:50:141
235002799278,5cyclictest0-21swapper/310:40:263
2350027992720,3cyclictest0-21swapper/311:45:273
2350027992720,3cyclictest0-21swapper/311:10:003
2350027992717,4cyclictest131rcu_preempt11:11:083
2350027992713,8cyclictest131rcu_preempt07:46:133
2350027992712,12cyclictest0-21swapper/307:20:153
235002799271,16cyclictest0-21swapper/312:30:373
2350027992710,14cyclictest0-21swapper/311:50:133
2350024992720,4cyclictest0-21swapper/208:30:002
2350024992719,4cyclictest0-21swapper/212:30:262
2350024992719,4cyclictest0-21swapper/210:45:182
2350024992718,6cyclictest0-21swapper/208:45:002
2350024992715,8cyclictest215-21dbus-daemon11:50:552
235001999276,4cyclictest131rcu_preempt09:10:121
2350019992721,3cyclictest0-21swapper/108:55:261
2350019992720,4cyclictest0-21swapper/112:00:271
2350019992719,6cyclictest146-21jbd2/mmcblk1p2-808:45:341
2350019992719,4cyclictest0-21swapper/112:35:001
2350019992718,6cyclictest231ktimers/108:05:141
235002799266,6cyclictest0-21swapper/310:45:193
235002799263,12cyclictest40-21ksoftirqd/309:10:233
2350027992615,7cyclictest2365176-21/usr/sbin/munin07:50:273
235002799261,17cyclictest0-21swapper/312:09:553
2350024992618,5cyclictest311ktimers/211:25:242
2350024992617,5cyclictest0-21swapper/209:05:192
2350019992621,3cyclictest2433235-21fschecks_count11:10:171
2350019992620,3cyclictest2436251-21kthreadd11:20:011
2350019992619,4cyclictest2392996-21kworker/u8:4+events_unbound09:20:021
2350019992619,4cyclictest0-21swapper/107:45:251
2350019992618,5cyclictest2436198-21kworker/u8:3+events_unbound11:30:521
2350019992618,4cyclictest0-21swapper/110:40:011
2350019992618,4cyclictest0-21swapper/107:55:101
2350019992617,6cyclictest0-21swapper/107:35:001
2350027992519,4cyclictest2429432-21sendmail11:00:013
2350027992517,4cyclictest0-21swapper/310:50:273
2350027992516,6cyclictest2378288-21tune2fs08:30:163
2350027992515,7cyclictest2359841-21/usr/sbin/munin07:35:253
2350027992514,7cyclictest2402506-21/usr/sbin/munin09:40:173
235002799251,18cyclictest0-21swapper/307:13:323
235002799251,15cyclictest0-21swapper/310:25:093
235002799251,14cyclictest0-21swapper/311:30:063
2350024992518,4cyclictest0-21swapper/210:15:272
2350024992516,6cyclictest0-21swapper/208:20:252
235001999258,3cyclictest131rcu_preempt09:25:021
2350019992519,4cyclictest2401674-21kworker/u8:2+events_unbound09:45:371
2350019992519,4cyclictest2351838-21latency_hist07:15:011
2350019992519,3cyclictest0-21swapper/111:50:271
2350019992519,3cyclictest0-21swapper/107:35:301
2350019992518,4cyclictest0-21swapper/111:40:281
2350019992515,7cyclictest2376456-21ls08:25:171
235002799240,12cyclictest0-21swapper/309:45:373
2350024992416,5cyclictest0-21swapper/211:20:262
2350019992418,3cyclictest0-21swapper/108:35:221
2350019992417,4cyclictest2410485-21idleruntime-cro10:05:011
2350019992417,3cyclictest0-21swapper/109:30:261
2350019992416,4cyclictest0-21swapper/112:20:011
2350019992416,4cyclictest0-21swapper/110:20:271
2350019992415,5cyclictest2446032-21grep11:45:301
2350019992414,5cyclictest2414078-21auditctl10:15:001
2350019992413,7cyclictest0-21swapper/110:35:001
2350024992317,3cyclictest0-21swapper/209:50:192
2350024992315,5cyclictest311ktimers/210:30:202
2350024992315,5cyclictest0-21swapper/209:55:232
2350024992315,5cyclictest0-21swapper/208:55:222
2350024992315,4cyclictest0-21swapper/209:50:002
2350024992315,4cyclictest0-21swapper/208:40:002
2350024992313,7cyclictest2395469-21diskstats09:20:152
2350019992317,3cyclictest0-21swapper/112:25:281
2350019992317,3cyclictest0-21swapper/108:30:251
2350019992316,5cyclictest146-21jbd2/mmcblk1p2-811:20:231
2350019992316,3cyclictest0-21swapper/112:15:011
2350019992314,6cyclictest2431140-21sh11:05:001
2350019992313,7cyclictest2458067-21grep12:20:261
2350019992312,8cyclictest2403633-21cron09:45:011
2350024992217,3cyclictest2411913-21unixbench_multi10:05:272
2350024992214,5cyclictest311ktimers/211:15:222
2350024992214,4cyclictest0-21swapper/212:20:002
2350024992213,6cyclictest2395135-21basename09:20:012
2350019992211,8cyclictest2439026-21cat11:25:271
2350024992114,5cyclictest2392996-21kworker/u8:4+events_unbound09:13:442
235002499211,3cyclictest2450153-21apt-get12:00:142
2350024992113,5cyclictest0-21swapper/208:45:242
2350024992113,5cyclictest0-21swapper/208:05:282
2350019992115,3cyclictest0-21swapper/111:55:261
2350019992115,3cyclictest0-21swapper/107:20:011
2350019992114,5cyclictest2369175-21kworker/u8:0+events_unbound08:10:271
2350019992112,6cyclictest2414593-21if_err_eth110:15:201
2350024992013,5cyclictest2398055-21unixbench_multi09:25:252
235001999201,4cyclictest0-21swapper/110:51:241
2350019991912,5cyclictest231ktimers/107:40:141
2350019991912,5cyclictest231ktimers/107:40:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional