You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-16 - 20:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Mon Jun 16, 2025 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7649992725,1cyclictest14423-21cut11:50:002
7645992422,1cyclictest583-21avahi-daemon08:00:001
7645992421,2cyclictest28750-21idleruntime-cro10:05:001
764599240,23cyclictest28120-21apt-get11:13:321
764599240,23cyclictest24571-21dump-pmu-power07:45:001
764099240,23cyclictest23943-21perf09:55:000
765399230,22cyclictest10139-21apt-get11:43:323
7649992321,1cyclictest593-21dbus-daemon08:50:002
7649992320,2cyclictest8609-21cron09:25:002
764999230,22cyclictest13147-21perf10:40:002
764999230,0cyclictest0-21swapper/207:38:302
764599230,22cyclictest6564-21grep08:15:001
764599230,22cyclictest21437-21cron08:45:001
764599230,22cyclictest18577-21apt-get08:40:001
764599230,0cyclictest0-21swapper/111:58:341
764099230,2cyclictest22063-21apt12:05:000
764099230,22cyclictest8328-21latency_hist10:30:000
764099230,1cyclictest17951-21munin-run10:50:000
7653992219,2cyclictest20236-21cmp07:38:313
765399220,22cyclictest0-21swapper/309:33:323
765399220,21cyclictest9511-21apt-get08:23:303
765399220,21cyclictest8133-21apt-get07:13:293
765399220,21cyclictest26980-21grep07:50:003
765399220,21cyclictest16073-21apt-get10:48:323
765399220,21cyclictest15025-21apt-get11:50:023
765399220,21cyclictest11909-21apt-get08:28:323
765399220,1cyclictest8979-21turbostat08:20:003
7649992220,1cyclictest3518-21dump-pmu-power10:20:002
7649992220,1cyclictest27608-21wc11:10:002
7649992220,1cyclictest0-21swapper/211:03:322
764999220,21cyclictest20860-21apt-get10:58:302
764999220,21cyclictest17202-21apt11:55:002
764999220,21cyclictest0-21swapper/211:18:312
764999220,21cyclictest0-21swapper/210:48:322
764999220,1cyclictest7239-21idleruntime-cro12:40:002
764999220,1cyclictest28732-21sh10:05:002
764999220,1cyclictest12393-21cron07:20:002
764999220,1cyclictest10726-21awk10:35:002
7645992220,1cyclictest13842-21perf09:35:001
7645992220,1cyclictest1001-21turbostat.cron09:10:011
764599220,21cyclictest0-21swapper/107:28:281
764599220,1cyclictest7721-21apt-get11:38:331
764599220,1cyclictest12429-21grep07:20:001
7640992220,1cyclictest8649-21munin-run09:25:000
764099220,21cyclictest593-21dbus-daemon08:30:000
764099220,21cyclictest20179-21apt-get07:38:320
764099220,21cyclictest17742-21apt-get07:33:310
764099220,21cyclictest10535-21apt-get07:18:310
764099220,21cyclictest0-21swapper/012:15:020
764099220,21cyclictest0-21swapper/011:08:320
764099220,21cyclictest0-21swapper/008:48:300
764099220,21cyclictest0-21swapper/008:33:290
764099220,1cyclictest11373-21grep08:25:000
7653992121,0cyclictest0-21swapper/308:53:333
7653992121,0cyclictest0-21swapper/308:53:333
7653992119,1cyclictest29644-21latency_hist12:20:003
765399210,21cyclictest0-21swapper/312:00:003
765399210,21cyclictest0-21swapper/307:18:313
765399210,20cyclictest6764-21apt-get09:23:333
765399210,20cyclictest32308-21apt-get08:03:293
765399210,20cyclictest0-21swapper/310:08:323
765399210,1cyclictest20348-21munin-run10:55:003
765399210,0cyclictest4688-21apt-get08:13:283
765399210,0cyclictest31659-21apt-get10:13:303
765399210,0cyclictest0-21swapper/309:43:313
765399210,0cyclictest0-21swapper/308:58:313
7649992119,1cyclictest4775-21sh11:30:002
7649992119,1cyclictest31026-21cron09:05:002
764999210,20cyclictest30190-21apt-get12:23:322
764999210,20cyclictest24456-21apt-get09:58:312
764999210,20cyclictest0-21swapper/210:13:302
764999210,0cyclictest0-21swapper/210:43:322
7645992121,0cyclictest0-21swapper/108:28:321
7645992120,1cyclictest8133-21apt-get07:13:291
7645992120,1cyclictest2296-21apt-get08:08:301
7645992120,1cyclictest1644-21apt-get10:18:321
7645992120,0cyclictest7086-21apt-get08:18:291
7645992120,0cyclictest4350-21apt-get09:18:341
7645992120,0cyclictest0-21swapper/107:48:291
7645992119,1cyclictest0-21swapper/111:30:001
764599210,20cyclictest0-21swapper/111:03:311
764599210,20cyclictest0-21swapper/110:23:311
764599210,20cyclictest0-21swapper/109:28:331
764599210,20cyclictest0-21swapper/107:53:281
764599210,20cyclictest0-21swapper/107:33:311
764599210,1cyclictest0-21swapper/110:58:301
764599210,0cyclictest0-21swapper/112:13:331
764599210,0cyclictest0-21swapper/107:38:311
7640992121,0cyclictest17727-21apt-get11:58:350
7640992121,0cyclictest0-21swapper/008:13:280
7640992120,1cyclictest0-21swapper/009:48:340
7640992120,0cyclictest0-21swapper/011:43:320
7640992119,1cyclictest9992-21cron07:15:000
7640992119,1cyclictest593-21dbus-daemon12:10:000
7640992119,1cyclictest0-21swapper/009:40:000
764099210,21cyclictest0-21swapper/012:28:340
764099210,20cyclictest0-21swapper/012:38:300
764099210,20cyclictest0-21swapper/011:33:310
764099210,20cyclictest0-21swapper/007:23:290
764099210,0cyclictest0-21swapper/010:53:360
764099210,0cyclictest0-21swapper/010:33:310
7653992020,0cyclictest27498-21apt-get07:53:283
7653992020,0cyclictest0-21swapper/311:08:323
7653992020,0cyclictest0-21swapper/310:38:323
7653992020,0cyclictest0-21swapper/310:23:313
7653992020,0cyclictest0-21swapper/308:08:303
7653992020,0cyclictest0-21swapper/307:28:283
7653992018,1cyclictest5315-21apt-get11:33:313
7653992018,1cyclictest31566-21apt-get09:08:333
765399200,0cyclictest0-21swapper/312:28:343
765399200,0cyclictest0-21swapper/309:58:303
765399200,0cyclictest0-21swapper/307:58:293
7649992020,0cyclictest29268-21apt-get10:08:322
7649992020,0cyclictest0-21swapper/211:33:312
7649992020,0cyclictest0-21swapper/208:03:302
7649992020,0cyclictest0-21swapper/207:43:292
7649992020,0cyclictest0-21swapper/207:33:302
7649992019,1cyclictest0-21swapper/209:13:322
7649992019,0cyclictest0-21swapper/212:08:362
7649992018,1cyclictest29656-21grep12:20:002
764999200,0cyclictest0-21swapper/208:08:302
7645992020,0cyclictest12948-21apt-get07:23:291
7645992020,0cyclictest0-21swapper/112:28:341
7645992020,0cyclictest0-21swapper/112:18:331
7645992020,0cyclictest0-21swapper/112:03:351
7645992020,0cyclictest0-21swapper/111:18:311
7645992020,0cyclictest0-21swapper/111:08:321
7645992020,0cyclictest0-21swapper/110:13:301
7645992019,0cyclictest0-21swapper/111:48:331
764599200,20cyclictest0-21swapper/112:23:321
764599200,20cyclictest0-21swapper/109:13:321
7640992020,0cyclictest15344-21apt-get07:28:280
7640992020,0cyclictest1007-21apt-get09:10:020
7640992020,0cyclictest0-21swapper/008:18:290
7640992020,0cyclictest0-21swapper/008:08:310
7640992020,0cyclictest0-21swapper/007:43:290
7640992018,1cyclictest268732sleep010:03:290
764099200,19cyclictest0-21swapper/010:18:340
764099200,0cyclictest4066-21apt-get10:23:310
764099200,0cyclictest0-21swapper/008:03:300
7653991919,0cyclictest6226-21sendmail-msp09:20:003
7653991919,0cyclictest0-21swapper/312:23:323
7653991919,0cyclictest0-21swapper/312:13:343
7653991919,0cyclictest0-21swapper/311:28:353
7653991919,0cyclictest0-21swapper/308:43:293
7649991919,0cyclictest0-21swapper/211:43:322
7649991919,0cyclictest0-21swapper/211:38:332
7649991919,0cyclictest0-21swapper/209:28:322
7649991919,0cyclictest0-21swapper/208:53:332
7649991919,0cyclictest0-21swapper/208:53:332
7649991919,0cyclictest0-21swapper/208:23:312
7649991919,0cyclictest0-21swapper/207:53:282
7649991919,0cyclictest0-21swapper/207:23:292
7649991918,0cyclictest20141-21apt-get12:03:352
7649991917,1cyclictest6576-21munin-run08:15:002
7645991919,0cyclictest15327-21apt-get11:53:311
7645991919,0cyclictest0-21swapper/110:08:321
7645991919,0cyclictest0-21swapper/109:48:341
7645991917,1cyclictest26241-21munin-run08:55:001
7645991917,1cyclictest26241-21munin-run08:55:001
7640991919,0cyclictest29172-21apt-get09:03:330
7640991919,0cyclictest16772-21apt-get09:43:320
7640991919,0cyclictest0-21swapper/010:43:320
7640991919,0cyclictest0-21swapper/009:18:340
7640991919,0cyclictest0-21swapper/009:08:330
7640991919,0cyclictest0-21swapper/008:45:000
7640991919,0cyclictest0-21swapper/008:38:310
7640991918,1cyclictest29268-21apt-get10:08:320
7640991918,1cyclictest0-21swapper/012:33:350
7640991917,1cyclictest31763-21turbostat.cron08:00:000
764099190,18cyclictest0-21swapper/011:23:310
764099190,0cyclictest0-21swapper/011:48:320
7653991818,0cyclictest0-21swapper/312:33:353
7653991818,0cyclictest0-21swapper/311:13:323
7653991818,0cyclictest0-21swapper/310:18:323
7653991818,0cyclictest0-21swapper/309:53:313
7653991816,1cyclictest10729-21cron10:35:003
7649991818,0cyclictest0-21swapper/211:23:302
7649991818,0cyclictest0-21swapper/210:28:322
7649991818,0cyclictest0-21swapper/209:54:322
7649991818,0cyclictest0-21swapper/209:33:322
7649991818,0cyclictest0-21swapper/208:33:292
7649991817,0cyclictest19213-21apt-get08:40:012
764999180,0cyclictest0-21swapper/209:08:332
7645991818,0cyclictest22592-21apt-get12:08:361
7645991818,0cyclictest14357-21apt-get09:38:321
7645991818,0cyclictest0-21swapper/112:33:351
7645991818,0cyclictest0-21swapper/110:53:361
7645991818,0cyclictest0-21swapper/108:33:291
7645991816,1cyclictest0-21swapper/110:45:001
7645991815,2cyclictest8614-21cron09:25:001
7640991818,0cyclictest0-21swapper/009:58:300
7640991816,1cyclictest0-21swapper/007:55:000
7653991717,0cyclictest0-21swapper/311:38:333
7653991717,0cyclictest0-21swapper/307:43:293
764999170,16cyclictest19567-21apt-get09:48:342
764999170,16cyclictest18662-21http09:45:002
7645991717,0cyclictest32308-21apt-get08:03:301
7645991717,0cyclictest0-21swapper/112:38:301
7645991717,0cyclictest0-21swapper/111:43:331
7645991717,0cyclictest0-21swapper/111:33:311
7645991717,0cyclictest0-21swapper/109:53:311
7645991714,1cyclictest313-21systemd-journal10:34:231
764599170,16cyclictest0-21swapper/110:48:421
7640991717,0cyclictest0-21swapper/011:53:310
7640991716,1cyclictest9166-21apt-get09:28:330
7640991716,1cyclictest23333-21apt-get11:03:320
7640991715,1cyclictest26837-21cron12:15:000
7640991715,1cyclictest1082-21cron10:15:000
7653991616,0cyclictest0-21swapper/312:38:303
7653991616,0cyclictest0-21swapper/310:28:333
7653991616,0cyclictest0-21swapper/309:48:343
7653991616,0cyclictest0-21swapper/309:38:323
765399160,15cyclictest31566-21ntp_states11:18:383
765399160,15cyclictest0-21swapper/312:01:333
7649991616,0cyclictest0-21swapper/210:23:312
7649991616,0cyclictest0-21swapper/209:18:342
7649991614,1cyclictest17220-21perf07:30:002
764999160,15cyclictest14520-21grep09:38:332
764999160,15cyclictest0-21swapper/207:58:302
764999160,0cyclictest0-21swapper/210:53:352
7645991616,0cyclictest0-21swapper/108:58:311
764599160,1cyclictest0-21swapper/110:40:001
764599160,1cyclictest0-21swapper/109:03:441
764599160,15cyclictest6428-21/usr/sbin/munin10:28:371
764599160,15cyclictest24172-21chrt09:55:141
7640991614,1cyclictest32412-21munin-run11:20:000
7640991614,1cyclictest27138-21cat08:58:320
7640991614,1cyclictest0-21swapper/011:28:360
7640991614,1cyclictest0-21swapper/009:30:010
764099160,15cyclictest30002-21perf11:15:000
764099160,0cyclictest0-21swapper/010:58:300
7653991515,0cyclictest0-21swapper/310:58:303
7653991515,0cyclictest0-21swapper/309:28:333
7653991515,0cyclictest0-21swapper/309:13:323
7653991515,0cyclictest0-21swapper/308:48:303
7653991515,0cyclictest0-21swapper/307:33:303
7653991514,1cyclictest16703-21apt-get08:38:323
7649991515,0cyclictest0-21swapper/208:28:322
7649991515,0cyclictest0-21swapper/207:48:292
7649991514,1cyclictest32586-21apt-get12:28:332
7649991514,0cyclictest2041-21apt-get12:30:012
7649991513,1cyclictest8941-21cron08:20:002
764599150,7cyclictest0-21swapper/109:43:311
7653991414,0cyclictest0-21swapper/311:46:023
7653991414,0cyclictest0-21swapper/310:02:093
765399140,14cyclictest0-21swapper/312:05:483
765399140,14cyclictest0-21swapper/311:20:343
765399140,14cyclictest0-21swapper/311:02:323
765399140,14cyclictest0-21swapper/310:40:143
765399140,14cyclictest0-21swapper/309:02:373
765399140,14cyclictest0-21swapper/308:33:193
765399140,14cyclictest0-21swapper/307:21:543
7649991414,0cyclictest0-21swapper/211:13:362
7649991414,0cyclictest0-21swapper/208:58:312
764999140,14cyclictest0-21swapper/212:10:252
764999140,14cyclictest0-21swapper/211:55:022
764999140,14cyclictest0-21swapper/207:11:002
7645991414,0cyclictest0-21swapper/108:20:091
764599140,14cyclictest0-21swapper/108:46:011
764599140,0cyclictest0-21swapper/111:20:581
764099140,14cyclictest0-21swapper/010:35:080
764099140,14cyclictest0-21swapper/007:46:260
764099140,0cyclictest0-21swapper/012:20:100
764099140,0cyclictest0-21swapper/011:36:410
7640991313,0cyclictest0-21swapper/008:50:060
7640991313,0cyclictest0-21swapper/008:50:060
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional