You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-25 - 21:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Nov 25, 2025 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30980992523,1cyclictest612-21gdbus07:25:003
30971992420,3cyclictest3898-21sh08:25:001
30967992422,1cyclictest3360-21sendmail-msp07:20:000
30980992321,1cyclictest14901-21cron11:00:003
3098099230,21cyclictest313-21systemd-journal08:15:013
3098099230,1cyclictest12506-21cron10:55:003
30976992320,2cyclictest18702-21sh07:50:002
3097699231,21cyclictest9859-21apt-get11:57:282
30971992321,1cyclictest0-21swapper/108:35:001
30971992320,2cyclictest16622-21dump-pmu-power12:10:001
3097199230,22cyclictest5236-21cron10:40:001
3097199230,22cyclictest21390-21cron12:20:001
30967992321,1cyclictest0-21swapper/010:40:000
3096799231,21cyclictest0-21swapper/011:35:010
3096799230,22cyclictest5775-21munin-run07:25:010
3096799230,22cyclictest31131-21perf08:15:010
3096799230,22cyclictest19010-21latency_hist12:15:000
30980992221,1cyclictest26851-21apt-get12:30:023
30980992220,1cyclictest26959-21awk11:25:003
30980992220,1cyclictest26959-21awk11:25:003
3098099220,21cyclictest593-21dbus-daemon09:10:013
3098099220,21cyclictest31651-21apt-get09:22:253
3098099220,21cyclictest19464-21apt-get12:17:273
3098099220,21cyclictest0-21swapper/309:15:003
3098099220,21cyclictest0-21swapper/307:47:293
3098099220,1cyclictest11832-21perf12:00:003
3098099220,1cyclictest0-21swapper/308:22:243
30976992220,1cyclictest9157-21dump-pmu-power08:35:002
30976992220,1cyclictest28523-21apt-get10:22:282
30976992220,1cyclictest0-21swapper/207:25:012
3097699220,21cyclictest31954-21apt-get08:17:252
3097699220,21cyclictest31397-21apt-get07:12:262
3097699220,21cyclictest26235-21turbostat.cron12:30:012
3097699220,21cyclictest1377-21apt-get07:17:252
3097699220,21cyclictest0-21swapper/210:32:262
3097699220,21cyclictest0-21swapper/208:27:262
3097699220,1cyclictest0-21swapper/209:00:002
30971992222,0cyclictest0-21swapper/111:12:271
30971992220,1cyclictest609-21polkitd12:30:011
30971992220,1cyclictest13935-21basename07:40:001
30971992219,2cyclictest4540-21cron11:45:001
3097199220,21cyclictest16060-21latency_hist09:55:011
3097199220,21cyclictest0-21swapper/111:02:281
3097199220,21cyclictest0-21swapper/107:17:271
3097199220,1cyclictest0-21swapper/110:30:001
30967992220,1cyclictest14953-21cat10:59:590
3096799220,21cyclictest0-21swapper/010:17:270
3096799220,21cyclictest0-21swapper/009:42:260
3096799220,21cyclictest0-21swapper/009:07:280
3096799220,1cyclictest19737-21awk11:10:000
3096799220,1cyclictest0-21swapper/012:30:020
30980992121,0cyclictest27401-21apt-get11:27:273
30980992120,1cyclictest19615-21apt-get08:57:273
30980992118,2cyclictest10134-21turbostat.cron10:50:013
3098099210,21cyclictest0-21swapper/309:57:273
3098099210,21cyclictest0-21swapper/308:15:023
3098099210,20cyclictest12029-21apt-get08:42:263
3098099210,20cyclictest0-21swapper/311:17:263
3098099210,20cyclictest0-21swapper/307:52:253
30976992120,0cyclictest11948-21apt-get07:37:282
30976992119,1cyclictest583-21avahi-daemon10:10:002
30976992119,1cyclictest17793-21apt-get11:07:272
30976992119,1cyclictest16040-21sh09:55:012
3097699210,21cyclictest0-21swapper/211:42:272
3097699210,20cyclictest7408-21apt-get11:52:272
3097699210,20cyclictest25654-21latency_hist10:15:002
3097699210,20cyclictest23973-21apt-get08:02:272
3097699210,20cyclictest0-21swapper/209:02:262
3097699210,20cyclictest0-21swapper/207:32:282
3097699210,0cyclictest0-21swapper/212:32:292
30971992120,1cyclictest12029-21apt-get08:42:261
30971992120,0cyclictest17064-21apt-get12:12:271
30971992119,2cyclictest0-21swapper/112:32:291
30971992119,1cyclictest0-21swapper/108:05:001
3097199210,21cyclictest0-21swapper/111:07:261
3097199210,21cyclictest0-21swapper/109:02:261
3097199210,21cyclictest0-21swapper/107:57:261
3097199210,21cyclictest0-21swapper/107:52:251
3097199210,20cyclictest30197-21apt-get11:32:271
3097199210,20cyclictest11546-21cron08:40:001
3097199210,20cyclictest0-21swapper/110:07:271
3097199210,0cyclictest5996-21apt-get09:35:021
3097199210,0cyclictest0-21swapper/110:57:251
3097199210,0cyclictest0-21swapper/109:42:271
3097199210,0cyclictest0-21swapper/109:22:251
3097199210,0cyclictest0-21swapper/109:07:271
30967992121,0cyclictest0-21swapper/008:07:270
30967992121,0cyclictest0-21swapper/007:47:280
30967992121,0cyclictest0-21swapper/007:27:270
30967992120,1cyclictest9158-21awk08:35:000
30967992119,1cyclictest11552-21sendmail08:40:000
30967992119,1cyclictest0-21swapper/008:02:270
3096799210,20cyclictest9056-21apt-get07:32:280
3096799210,20cyclictest18912-21apt-get10:02:270
3096799210,1cyclictest2610-21apt-get11:42:270
3096799210,0cyclictest16498-21apt-get09:57:260
3096799210,0cyclictest0-21swapper/007:37:270
30980992020,0cyclictest20189-21apt-get11:12:273
30980992020,0cyclictest12274-21apt-get12:02:263
30980992020,0cyclictest0-21swapper/312:12:273
30980992020,0cyclictest0-21swapper/310:42:253
30980992020,0cyclictest0-21swapper/310:02:263
30980992020,0cyclictest0-21swapper/308:47:263
30980992020,0cyclictest0-21swapper/308:32:293
30980992020,0cyclictest0-21swapper/308:02:273
30980992017,2cyclictest5289-21dump-pmu-power10:39:593
3098099200,0cyclictest0-21swapper/311:52:283
3098099200,0cyclictest0-21swapper/310:12:263
3098099200,0cyclictest0-21swapper/310:07:263
30976992020,0cyclictest4041-21apt-get09:32:292
30976992020,0cyclictest0-21swapper/211:22:262
30976992020,0cyclictest0-21swapper/211:22:262
30976992020,0cyclictest0-21swapper/210:42:262
30976992020,0cyclictest0-21swapper/209:22:252
30976992019,1cyclictest0-21swapper/209:47:262
30976992019,0cyclictest0-21swapper/212:12:262
30971992020,0cyclictest10578-21apt-get10:52:261
30971992020,0cyclictest0-21swapper/111:57:281
30971992020,0cyclictest0-21swapper/111:37:281
30971992020,0cyclictest0-21swapper/110:42:261
30971992020,0cyclictest0-21swapper/110:22:281
30971992020,0cyclictest0-21swapper/109:32:281
30971992020,0cyclictest0-21swapper/108:57:271
30971992020,0cyclictest0-21swapper/108:12:261
30971992020,0cyclictest0-21swapper/107:25:001
3097199200,20cyclictest0-21swapper/109:57:271
3097199200,20cyclictest0-21swapper/108:15:021
3097199200,0cyclictest0-21swapper/109:17:251
30967992020,0cyclictest0-21swapper/012:22:270
30967992020,0cyclictest0-21swapper/012:02:260
30967992020,0cyclictest0-21swapper/010:47:280
30967992020,0cyclictest0-21swapper/010:27:250
30967992020,0cyclictest0-21swapper/009:47:260
30967992020,0cyclictest0-21swapper/009:32:280
30967992020,0cyclictest0-21swapper/007:12:260
30967992018,1cyclictest8836-21cron09:40:000
30967992018,1cyclictest21107-21sh07:54:590
3096799200,20cyclictest0-21swapper/012:07:260
3096799200,20cyclictest0-21swapper/011:12:270
3096799200,0cyclictest0-21swapper/011:37:280
3096799200,0cyclictest0-21swapper/009:12:280
30980991919,0cyclictest9307-21apt-get09:42:263
30980991919,0cyclictest0-21swapper/311:07:273
30980991919,0cyclictest0-21swapper/309:17:263
30980991918,1cyclictest6206-21apt-get07:27:263
30980991918,0cyclictest0-21swapper/310:22:283
3098099190,0cyclictest0-21swapper/308:27:273
30976991919,0cyclictest0-21swapper/207:42:252
30976991918,1cyclictest12977-21apt-get10:57:252
30976991917,1cyclictest0-21swapper/211:35:012
30976991917,1cyclictest0-21swapper/209:57:272
30976991917,1cyclictest0-21swapper/208:00:002
3097699190,18cyclictest26105-21apt-get10:17:272
3097699190,15cyclictest0-21swapper/212:22:262
30971991919,0cyclictest25004-21apt-get11:22:271
30971991919,0cyclictest25004-21apt-get11:22:271
30971991919,0cyclictest0-21swapper/108:07:271
30971991918,1cyclictest26830-21apt-get09:12:271
30971991918,0cyclictest14364-21apt-get07:42:251
3097199190,2cyclictest8138-21latency_hist07:30:011
30967991919,0cyclictest0-21swapper/012:37:250
30967991919,0cyclictest0-21swapper/011:02:270
30967991917,1cyclictest31208-21munin-run09:20:000
3096799190,18cyclictest0-21swapper/011:52:270
30980991818,0cyclictest0-21swapper/312:22:253
30980991818,0cyclictest0-21swapper/308:07:273
30980991818,0cyclictest0-21swapper/307:32:283
30980991817,0cyclictest0-21swapper/311:32:273
30976991818,0cyclictest22586-21apt-get11:17:252
30976991818,0cyclictest0-21swapper/211:27:282
30976991818,0cyclictest0-21swapper/211:12:282
30976991818,0cyclictest0-21swapper/209:42:272
30976991817,1cyclictest19154-21apt-get07:52:252
30976991816,1cyclictest0-21swapper/208:45:022
30971991818,0cyclictest17218-21apt-get08:52:281
30971991818,0cyclictest17218-21apt-get08:52:281
30971991818,0cyclictest0-21swapper/110:12:271
30971991818,0cyclictest0-21swapper/108:45:021
30971991818,0cyclictest0-21swapper/108:27:271
30971991816,1cyclictest6945-21cron11:50:001
30967991818,0cyclictest22031-21apt-get09:02:260
30967991818,0cyclictest0-21swapper/009:22:250
30967991817,1cyclictest0-21swapper/010:07:260
30967991817,0cyclictest4367-21apt-get08:27:270
30967991816,1cyclictest26219-21apt-get12:30:010
30980991717,0cyclictest0-21swapper/311:37:283
30980991717,0cyclictest0-21swapper/307:35:013
30980991717,0cyclictest0-21swapper/307:12:263
30980991715,1cyclictest16329-21munin-run07:45:003
30980991715,1cyclictest0-21swapper/311:45:003
30976991717,0cyclictest28319-21apt-get08:10:032
30976991717,0cyclictest0-21swapper/212:37:262
30976991717,0cyclictest0-21swapper/212:07:262
30976991717,0cyclictest0-21swapper/212:02:262
30976991717,0cyclictest0-21swapper/210:37:272
30976991717,0cyclictest0-21swapper/209:35:022
3097699170,1cyclictest0-21swapper/208:22:242
3097699170,0cyclictest0-21swapper/209:07:282
30971991717,0cyclictest0-21swapper/112:02:241
30967991717,0cyclictest5733-21apt-get10:42:260
30967991717,0cyclictest0-21swapper/012:17:270
30967991717,0cyclictest0-21swapper/011:57:280
30967991717,0cyclictest0-21swapper/011:22:270
30967991717,0cyclictest0-21swapper/011:22:270
30967991717,0cyclictest0-21swapper/010:52:250
30967991717,0cyclictest0-21swapper/009:27:260
30967991717,0cyclictest0-21swapper/008:22:250
30967991715,1cyclictest1510-21sendmail-msp08:20:010
30980991616,0cyclictest0-21swapper/312:07:263
30980991616,0cyclictest0-21swapper/310:17:273
30980991616,0cyclictest0-21swapper/309:32:283
30980991614,1cyclictest31438-21sendmail12:40:003
30980991614,1cyclictest0-21swapper/309:02:323
3098099160,15cyclictest0-21swapper/312:27:283
30976991614,1cyclictest28319-21apt-get08:10:002
30976991614,1cyclictest0-21swapper/209:17:282
3097699160,15cyclictest19816-21memory10:02:312
30971991616,0cyclictest0-21swapper/109:47:261
30971991615,0cyclictest0-21swapper/110:17:261
3096799162,2cyclictest20677-21grep08:57:310
30967991616,0cyclictest0-21swapper/007:42:240
30976991515,0cyclictest5010-21apt-get11:47:272
30976991515,0cyclictest0-21swapper/210:47:282
30976991515,0cyclictest0-21swapper/208:52:282
30976991515,0cyclictest0-21swapper/208:52:282
30971991515,0cyclictest0-21swapper/107:12:261
3097199150,1cyclictest0-21swapper/112:37:271
3097199150,14cyclictest0-21swapper/109:27:351
30967991515,0cyclictest0-21swapper/009:52:260
30967991514,1cyclictest0-21swapper/010:32:330
30967991512,2cyclictest30444-21cron10:25:000
30967991512,2cyclictest19163-21latency_hist08:55:000
30967991512,2cyclictest19163-21latency_hist08:54:590
30980991414,0cyclictest0-21swapper/310:30:253
30980991414,0cyclictest0-21swapper/309:52:263
30980991414,0cyclictest0-21swapper/308:52:283
30980991414,0cyclictest0-21swapper/308:52:283
3098099140,14cyclictest0-21swapper/311:45:233
3098099140,14cyclictest0-21swapper/311:01:273
3098099140,14cyclictest0-21swapper/310:25:193
3098099140,14cyclictest0-21swapper/309:28:063
3098099140,14cyclictest0-21swapper/308:35:133
3098099140,14cyclictest0-21swapper/307:56:423
3098099140,14cyclictest0-21swapper/307:16:003
3098099140,0cyclictest0-21swapper/309:47:323
3098099140,0cyclictest0-21swapper/309:35:313
30976991414,0cyclictest0-21swapper/211:35:272
3097699140,14cyclictest0-21swapper/212:16:442
3097699140,14cyclictest0-21swapper/211:00:212
3097699140,14cyclictest0-21swapper/210:50:472
3097699140,14cyclictest0-21swapper/210:28:042
3097699140,14cyclictest0-21swapper/209:14:052
3097699140,14cyclictest0-21swapper/208:40:312
3097699140,14cyclictest0-21swapper/208:35:202
3097699140,14cyclictest0-21swapper/207:26:522
30971991414,0cyclictest0-21swapper/107:45:081
3097199140,14cyclictest0-21swapper/110:45:131
3097199140,14cyclictest0-21swapper/110:30:501
3097199140,14cyclictest0-21swapper/107:30:061
3097199140,0cyclictest0-21swapper/112:21:091
3097199140,0cyclictest0-21swapper/111:25:241
3096799140,14cyclictest0-21swapper/011:45:520
3096799140,14cyclictest0-21swapper/011:25:240
3096799140,14cyclictest0-21swapper/010:11:000
3096799140,14cyclictest0-21swapper/008:45:260
3096799140,0cyclictest0-21swapper/008:41:050
3096799140,0cyclictest0-21swapper/007:55:150
30976991313,0cyclictest0-21swapper/209:26:102
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional