You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-09 - 19:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Tue Dec 09, 2025 12:39:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4077783995443,6cyclictest3227738-21cups-browsed12:09:272
4077783995140,8cyclictest4154763-21aten_r2power_cu10:14:132
407778299493,39cyclictest3227738-21cups-browsed11:14:251
4077781994938,7cyclictest30930-21ssh11:54:290
4077783994834,11cyclictest4113773-21idleruntime-cro09:10:012
4077781994834,9cyclictest12450irq/32-eth012:34:280
4077781994834,9cyclictest12450irq/32-eth012:34:280
4077782994736,8cyclictest4097280-21systemd-run08:15:021
4077781994726,15cyclictest4089990-40mandb07:50:330
4077781994726,15cyclictest4089990-40mandb07:50:330
4077783994539,3cyclictest0-21swapper/212:40:012
4077783994539,3cyclictest0-21swapper/212:40:012
4077783994534,7cyclictest1-21systemd10:25:022
407778299452,36cyclictest4179248-21ssh10:49:281
4077781994538,3cyclictest0-21swapper/009:10:010
4077781994535,5cyclictest0-21swapper/009:19:220
4077783994434,6cyclictest4101789-21systemd-run08:30:012
4077783994336,4cyclictest3607740-21kworker/u8:12-xprtiod09:17:012
4077782994338,2cyclictest0-21swapper/109:30:011
407778199434,31cyclictest4158821-21ssh10:19:220
4077781994321,11cyclictest4113765-21diskmemload11:29:320
407778399426,30cyclictest3182-21ssh11:14:192
4077783994234,5cyclictest4172385-21ntp_kernel_pll_10:39:262
4077782994233,5cyclictest45920-21munin-run12:15:011
4077782994232,7cyclictest4152722-21head10:09:361
4077782994232,7cyclictest4152722-21head10:09:361
4077783994135,3cyclictest0-21swapper/210:20:012
4077783994132,6cyclictest56186-21systemd-run12:30:022
4077783994130,8cyclictest4133543-21sendmail09:40:012
4077783994130,8cyclictest4133543-21sendmail09:40:012
4077782994132,4cyclictest0-21swapper/110:34:221
4077781994134,5cyclictest4094812-21fschecks_count08:09:150
4077781994123,11cyclictest12450irq/32-eth008:39:270
4077781994029,7cyclictest422-21dbus-daemon10:23:370
4077781994029,7cyclictest27899-21latency_hist11:50:020
4077781994027,9cyclictest4156275-21munin-plugin-st10:15:000
4077783993935,2cyclictest0-21swapper/210:00:012
4077783993935,2cyclictest0-21swapper/210:00:012
4077783993932,3cyclictest0-21swapper/209:45:002
4077783993931,5cyclictest422-21dbus-daemon09:50:012
4077783993929,7cyclictest421-21avahi-daemon11:39:222
4077782993930,5cyclictest4100315-21latency_hist08:25:021
4077781993932,4cyclictest4082816-21uniq07:29:170
4077781993931,6cyclictest7744-21idleruntime-cro11:20:010
4077781993926,5cyclictest4152303-21ssh10:09:310
4077781993926,5cyclictest4152303-21ssh10:09:310
4077783993830,5cyclictest3227738-21cups-browsed11:09:382
4077783993829,5cyclictest3227738-21cups-browsed11:52:582
407778299384,26cyclictest3227706-21cupsd11:44:251
4077781993827,3cyclictest151ktimers/010:34:200
407778399377,24cyclictest4129600-21ssh09:34:282
4077783993731,3cyclictest0-21swapper/209:00:012
4077783993728,6cyclictest453-21systemd-logind10:30:012
407778399372,29cyclictest34458-21ssh11:59:242
4077782993729,5cyclictest7742-21munin-plugin-st11:20:011
4077782993729,5cyclictest4159078-21processes10:19:301
4077782993728,6cyclictest422-21dbus-daemon07:50:301
4077782993728,6cyclictest422-21dbus-daemon07:50:301
4077782993728,6cyclictest4092752-21cron08:00:001
4077782993728,6cyclictest3227738-21cups-browsed10:14:141
4077781993730,4cyclictest4123903-21unixbench_singl09:24:470
4077781993730,3cyclictest0-21swapper/012:09:270
4077781993713,16cyclictest12450irq/32-eth011:34:210
407778399363,26cyclictest4116634-21ssh09:14:222
4077783993628,5cyclictest4109282-21systemd-run08:55:012
4077783993625,8cyclictest4176642-21/usr/sbin/munin10:45:022
4077782993626,6cyclictest266-21systemd-journal08:30:011
4077782993623,9cyclictest17943-21unixbench_singl11:34:341
4077781993630,3cyclictest4187186-21idleruntime-cro11:00:010
4077781993626,7cyclictest657-21snmpd12:39:230
4077781993626,7cyclictest657-21snmpd12:39:230
407778199362,27cyclictest51922-21ssh12:24:280
4077783993529,3cyclictest0-21swapper/212:19:412
4077783993529,3cyclictest0-21swapper/210:05:012
4077783993526,6cyclictest421-21avahi-daemon11:01:102
4077783993526,6cyclictest4141823-21aten_r2power_en09:54:102
4077783993526,6cyclictest4141823-21aten_r2power_en09:54:102
4077782993528,3cyclictest0-21swapper/109:49:271
4077782993526,6cyclictest422-21dbus-daemon09:40:011
4077782993526,6cyclictest422-21dbus-daemon09:40:011
4077781993529,3cyclictest0-21swapper/010:29:240
4077781993528,3cyclictest0-21swapper/009:50:000
4077781993527,5cyclictest55592-21sendmail_mailtr12:29:320
4077781993526,6cyclictest0-21swapper/011:40:030
4077781993525,8cyclictest4096308-21fschecks_count08:14:150
4077781993525,6cyclictest421-21avahi-daemon11:58:040
4077783993426,5cyclictest4080809-21systemd-run07:20:022
4077783993426,5cyclictest27719-21idleruntime-cro11:50:002
4077783993425,6cyclictest4123510-21sendmail_mailst09:24:422
4077783993425,5cyclictest4125971-21sh09:29:162
4077783993424,8cyclictest4186758-21timerandwakeup10:59:362
4077783993424,7cyclictest38060-21ntp_kernel_pll_12:04:222
4077782993428,3cyclictest0-21swapper/111:40:011
4077782993426,5cyclictest61800-21fschecks_time12:39:151
4077782993426,5cyclictest61800-21fschecks_time12:39:151
4077782993424,7cyclictest4113765-21diskmemload09:35:011
407778299342,23cyclictest4189980-21ssh11:04:241
407778199343,25cyclictest3262-21ssh11:14:220
4077781993425,6cyclictest3227738-21cups-browsed10:41:040
4077781993423,7cyclictest3227738-21cups-browsed09:36:110
4077781993423,7cyclictest3227738-21cups-browsed09:36:110
4077781993412,5cyclictest14-21ksoftirqd/011:04:240
4077783993326,4cyclictest4169784-21systemd-run10:35:012
4077783993326,3cyclictest0-21swapper/207:14:252
4077783993325,5cyclictest59503-21cron12:35:012
4077783993325,5cyclictest59503-21cron12:35:012
4077783993324,6cyclictest4099326-21fschecks_count08:24:162
4077783993322,8cyclictest4093701-21ntp_kernel_pll_08:04:252
407778399332,24cyclictest13832-21ssh11:29:262
4077782993327,3cyclictest27746-21kworker/u8:14-nfsiod12:10:011
4077782993327,3cyclictest0-21swapper/110:30:011
4077782993324,6cyclictest491-21gdbus08:39:151
4077782993322,8cyclictest4182978-21switchtime10:54:331
4077781993326,5cyclictest422-21dbus-daemon10:00:000
4077781993326,5cyclictest422-21dbus-daemon10:00:000
4077781993324,6cyclictest0-21swapper/011:24:130
4077781993322,8cyclictest0-21swapper/009:35:010
4077781993321,3cyclictest0-21swapper/009:29:150
4077781993320,8cyclictest4179258-21ntpq10:49:280
4077781993316,13cyclictest48299-21ntpq12:19:240
4077781993316,11cyclictest4078674-21ntpq07:14:220
407778199331,27cyclictest4193826-21ssh11:09:330
4077783993224,5cyclictest4077786-21systemd-run07:10:012
407778399322,26cyclictest7052-21ssh11:19:312
407778299323,22cyclictest48307-21sshd12:19:241
4077782993226,3cyclictest0-21swapper/109:55:011
4077782993226,3cyclictest0-21swapper/109:55:011
4077782993226,3cyclictest0-21swapper/109:15:011
4077782993225,4cyclictest4136514-21tail09:44:411
4077782993223,7cyclictest51855-21sh12:24:261
4077782993223,6cyclictest4103256-21cron08:35:011
407778299322,25cyclictest10403-21ssh11:24:231
4077782993221,7cyclictest3227738-21cups-browsed11:25:181
4077782993219,9cyclictest59021-21switchtime12:34:371
4077782993219,9cyclictest59021-21switchtime12:34:371
4077782993219,9cyclictest4172270-21needreboot10:39:221
407778299321,24cyclictest4162224-21ssh10:24:231
4077781993225,3cyclictest0-21swapper/010:04:350
4077781993224,6cyclictest698-21cron11:45:010
4077781993224,6cyclictest4100296-21sh08:25:010
4077781993224,4cyclictest0-21swapper/009:45:000
4077781993223,7cyclictest4106291-21sh08:45:010
4077781993222,5cyclictest4112501-21kworker/u8:0-xprtiod09:12:290
4077781993221,5cyclictest4107145-21ntpq08:49:220
4077781993221,5cyclictest4107145-21ntpq08:49:220
4077781993218,4cyclictest0-21swapper/008:00:010
4077781993212,6cyclictest14-21ksoftirqd/010:54:100
407778399314,12cyclictest4113765-21diskmemload10:46:022
4077783993122,6cyclictest4182311-21/usr/sbin/munin10:54:172
4077783993122,5cyclictest4085204-21unixbench_singl07:34:342
4077782993125,3cyclictest0-21swapper/109:59:291
4077782993125,3cyclictest0-21swapper/109:59:291
4077782993123,5cyclictest4094818-21fschecks_count08:09:151
4077782993123,5cyclictest4084310-21fschecks_count07:34:141
4077782993120,8cyclictest4148258-21cpu10:04:121
4077782993116,8cyclictest38085-21ssh12:04:221
4077781993120,8cyclictest4101188-21ntpq08:29:200
4077781993115,10cyclictest4108672-21ntpq08:54:230
4077781993112,4cyclictest14-21ksoftirqd/008:59:250
4077783993023,3cyclictest0-21swapper/208:45:012
4077783993022,4cyclictest0-21swapper/211:24:202
4077782993023,3cyclictest0-21swapper/111:50:021
4077782993023,3cyclictest0-21swapper/111:09:301
4077782993022,5cyclictest4088265-21cron07:45:011
4077782993015,11cyclictest422-21dbus-daemon10:41:481
407778299301,22cyclictest30969-21ssh11:54:311
4077781993022,6cyclictest4142693-21mailstats09:54:320
4077781993022,6cyclictest4142693-21mailstats09:54:320
4077781993022,4cyclictest4112176-21unixbench_multi09:04:330
407778199302,21cyclictest421-21avahi-daemon07:44:240
4077781993020,7cyclictest4098100-21needreboot08:19:200
4077781993016,8cyclictest4093712-21ntpq08:04:260
4077783992922,4cyclictest956-21mta-sts-daemon11:44:412
4077783992921,5cyclictest421-21avahi-daemon12:21:532
4077783992918,8cyclictest4104152-21ntp_kernel_pll_08:39:222
407778299295,15cyclictest4120111-21ssh09:19:261
4077782992922,4cyclictest4089657-21tail07:49:351
407778299292,19cyclictest55393-21sshd12:29:261
4077782992920,7cyclictest4098758-21cron08:20:011
4077782992920,6cyclictest1-21systemd08:05:011
4077782992920,6cyclictest1-21systemd07:40:011
4077781992923,3cyclictest0-21swapper/010:40:010
4077781992922,4cyclictest4086812-21latency_hist07:40:020
4077781992916,8cyclictest4084770-21ntpq07:34:280
4077781992916,8cyclictest4081758-21ntpq07:24:260
407778399284,11cyclictest4113765-21diskmemload12:11:592
4077783992822,2cyclictest0-21swapper/211:34:202
407778299285,13cyclictest4123317-21ssh09:24:321
4077782992822,3cyclictest0-21swapper/110:59:131
4077782992821,4cyclictest956-21mta-sts-daemon09:07:251
407778199288,5cyclictest14-21ksoftirqd/012:03:010
4077781992816,8cyclictest0-21swapper/012:14:350
4077781992815,8cyclictest4102641-21ntpq08:34:220
4077781992814,11cyclictest4079857-21/usr/sbin/munin07:19:160
4077783992717,7cyclictest4103413-21latency_hist08:35:022
4077782992720,4cyclictest0-21swapper/111:59:251
4077782992719,5cyclictest453-21systemd-logind07:15:021
4077782992719,5cyclictest4110707-21head08:59:361
4077781992720,5cyclictest3227738-21cups-browsed07:46:020
407778399263,15cyclictest657-21snmpd07:49:232
4077783992620,3cyclictest0-21swapper/207:40:012
4077783992618,5cyclictest4082185-21unixbench_multi07:24:332
4077783992611,11cyclictest599-21lldpd10:06:342
4077783992611,11cyclictest599-21lldpd10:06:342
4077782992618,5cyclictest4109292-21latency_hist08:55:011
4077782992617,6cyclictest421-21avahi-daemon07:29:331
4077782992616,7cyclictest4105007-21sshd08:41:291
4077783992519,4cyclictest959-21mta-sts-daemon07:25:182
4077783992419,2cyclictest0-21swapper/207:44:342
4077783992418,4cyclictest4077780-21cyclictest08:14:132
4077783992418,4cyclictest0-21swapper/207:50:322
4077783992418,4cyclictest0-21swapper/207:50:322
4077783992417,4cyclictest959-21mta-sts-daemon08:19:012
4077783992417,4cyclictest959-21mta-sts-daemon08:09:252
4077783992417,4cyclictest4092762-21munin-run08:00:002
4077783992416,5cyclictest4112295-21systemd-run09:05:022
4077782992417,4cyclictest957-21mta-sts-daemon08:49:201
4077782992417,4cyclictest957-21mta-sts-daemon08:49:201
4077782992417,4cyclictest4080766-21ls07:20:011
4077782992415,6cyclictest1-21systemd07:10:011
4077783992312,8cyclictest1690079-21ntpd08:49:242
4077783992312,8cyclictest1690079-21ntpd08:49:242
4077782992315,5cyclictest4111463-21latency09:04:171
4077782992213,6cyclictest4082278-21latency_hist07:25:011
407778199207,9cyclictest4077789-21systemd-run07:10:010
407778499163,8cyclictest0-21swapper/311:54:293
407778499163,7cyclictest0-21swapper/307:50:343
407778499163,7cyclictest0-21swapper/307:50:343
407778499163,6cyclictest0-21swapper/309:44:293
407778499162,7cyclictest0-21swapper/309:34:233
407778499162,7cyclictest0-21swapper/307:19:273
407778499161,9cyclictest0-21swapper/312:29:263
407778499161,9cyclictest0-21swapper/311:14:253
407778499154,6cyclictest0-21swapper/309:54:303
407778499154,6cyclictest0-21swapper/309:54:303
407778499153,7cyclictest0-21swapper/311:19:233
407778499153,6cyclictest0-21swapper/311:09:263
407778499152,8cyclictest0-21swapper/310:25:023
407778499152,7cyclictest0-21swapper/312:39:103
407778499152,7cyclictest0-21swapper/312:39:103
407778499152,7cyclictest0-21swapper/311:50:013
407778499152,7cyclictest0-21swapper/311:44:263
407778499152,7cyclictest0-21swapper/310:49:303
407778499152,7cyclictest0-21swapper/310:34:203
407778499152,6cyclictest0-21swapper/307:44:223
407778499151,8cyclictest0-21swapper/312:34:243
407778499151,8cyclictest0-21swapper/312:34:243
407778499151,8cyclictest0-21swapper/310:04:283
407778499151,8cyclictest0-21swapper/309:29:293
407778499151,7cyclictest0-21swapper/311:24:233
407778499151,7cyclictest0-21swapper/309:19:203
407778499151,7cyclictest0-21swapper/308:49:283
407778499151,7cyclictest0-21swapper/308:49:283
407778499151,7cyclictest0-21swapper/308:04:243
407778499151,6cyclictest0-21swapper/308:54:213
407778499151,6cyclictest0-21swapper/308:19:233
407778499151,6cyclictest0-21swapper/308:09:233
407778499151,6cyclictest0-21swapper/307:29:243
407778499143,8cyclictest0-21swapper/312:14:213
407778499143,7cyclictest0-21swapper/311:34:233
407778499143,6cyclictest0-21swapper/312:24:223
407778499142,7cyclictest0-21swapper/311:04:253
407778499142,7cyclictest0-21swapper/310:29:203
407778499142,7cyclictest0-21swapper/310:09:243
407778499142,7cyclictest0-21swapper/310:09:243
407778499141,8cyclictest0-21swapper/309:14:233
407778499141,7cyclictest0-21swapper/311:29:293
407778499141,7cyclictest0-21swapper/310:44:263
407778499141,7cyclictest0-21swapper/310:14:253
407778499141,6cyclictest0-21swapper/307:59:223
407778499141,6cyclictest0-21swapper/307:24:253
407778499141,5cyclictest0-21swapper/308:39:203
407778499141,4cyclictest0-21swapper/310:19:223
407778499133,8cyclictest0-21swapper/312:04:233
407778499133,4cyclictest0-21swapper/307:39:283
407778499132,7cyclictest0-21swapper/312:19:113
407778499132,7cyclictest0-21swapper/309:49:243
407778499132,6cyclictest0-21swapper/311:37:303
407778499132,6cyclictest0-21swapper/307:49:253
407778499132,5cyclictest0-21swapper/307:14:263
407778499131,7cyclictest0-21swapper/310:39:183
407778499131,7cyclictest0-21swapper/309:58:293
407778499131,7cyclictest0-21swapper/309:58:293
407778499131,7cyclictest0-21swapper/309:10:013
407778499131,7cyclictest0-21swapper/307:34:253
407778499131,6cyclictest0-21swapper/309:04:243
407778499131,6cyclictest0-21swapper/308:24:233
407778499131,6cyclictest0-21swapper/308:14:243
407778499123,7cyclictest0-21swapper/312:09:113
407778499122,7cyclictest0-21swapper/309:39:113
407778499122,7cyclictest0-21swapper/309:39:113
407778499122,6cyclictest0-21swapper/310:59:153
407778499122,5cyclictest0-21swapper/311:59:223
407778499121,6cyclictest0-21swapper/310:51:583
407778499114,5cyclictest0-21swapper/308:44:273
407778499111,6cyclictest0-21swapper/309:24:213
407778499111,6cyclictest0-21swapper/308:34:233
407778499101,6cyclictest0-21swapper/308:59:153
407778499100,5cyclictest0-21swapper/308:29:193
40777849961,2cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional