You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-27 - 09:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed Aug 27, 2025 00:40:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1465099585,8cyclictest171rcu_preempt22:08:271
1465099585,8cyclictest171rcu_preempt22:08:271
1465199517,34cyclictest136038-21ssh23:08:332
14650994942,4cyclictest179915-21kworker/u8:12-xprtiod00:18:301
14650994839,5cyclictest124476-21/usr/sbin/munin22:53:141
14650994739,5cyclictest108399-21systemd-run22:30:021
14649994634,10cyclictest161487-21wc23:43:340
14650994536,6cyclictest50638-21diskmemload23:43:321
14651994436,5cyclictest97618-21latency_hist22:15:022
14651994436,5cyclictest97618-21latency_hist22:15:022
14651994435,6cyclictest1-21systemd23:05:012
14651994433,6cyclictest0-21swapper/222:18:222
14650994435,4cyclictest0-21swapper/121:13:291
14650994435,4cyclictest0-21swapper/121:13:291
14650994435,4cyclictest0-21swapper/100:28:291
14650994432,8cyclictest2458627-21cups-browsed21:44:521
14649994437,4cyclictest151730-21ssh23:30:010
14651994334,6cyclictest162709-21latency_hist23:45:022
14650994334,6cyclictest164859-21ntp_kernel_err23:48:281
14649994333,6cyclictest72326-21ssh21:40:010
14651994233,6cyclictest82482-21unixbench_singl21:53:392
14650994230,9cyclictest182382-21unixbench_multi00:13:431
14649994236,4cyclictest108413-21latency_hist22:30:020
14649994233,6cyclictest2458627-21cups-browsed23:38:220
14651994131,7cyclictest153808-21kernelversion23:33:192
14650994132,6cyclictest50638-21diskmemload23:23:211
14650994132,5cyclictest119188-21kworker/u8:12-rpciod23:25:071
14649994119,15cyclictest12450irq/32-eth000:38:300
14651994032,3cyclictest0-21swapper/222:00:012
14651994031,4cyclictest0-21swapper/200:00:022
14651994031,4cyclictest0-21swapper/200:00:022
14651994030,6cyclictest2458627-21cups-browsed22:30:032
14650994031,5cyclictest172639-21unixbench_singl23:58:361
14650994031,5cyclictest172639-21unixbench_singl23:58:361
14650994031,5cyclictest115569-21idleruntime-cro22:40:011
14650994029,7cyclictest81964-21ntp_states21:53:311
14649994028,10cyclictest187673-21/usr/sbin/munin00:23:190
14649994025,9cyclictest151ktimers/020:13:280
1465199393,31cyclictest76571-21rm21:46:022
14651993930,6cyclictest2458627-21cups-browsed23:20:422
1465099394,28cyclictest50638-21diskmemload23:08:261
14650993930,5cyclictest41628-21munin-plugin-st20:40:011
14650993929,5cyclictest0-21swapper/119:38:281
1464999394,29cyclictest657-21snmpd23:53:230
14649993932,4cyclictest179011-21sendmail_mailtr00:08:330
14649993932,4cyclictest179011-21sendmail_mailtr00:08:330
14649993931,5cyclictest2458627-21cups-browsed21:51:180
14649993917,14cyclictest151ktimers/023:18:260
14651993830,5cyclictest50638-21diskmemload23:40:022
14651993829,4cyclictest0-21swapper/222:53:292
1465099384,29cyclictest85507-21ssh21:58:301
14650993829,6cyclictest57443-21unixbench_singl21:18:371
14650993829,6cyclictest57443-21unixbench_singl21:18:371
14650993829,5cyclictest101134-21cron22:20:011
14650993826,9cyclictest67503-21ntpq21:33:211
14649993830,5cyclictest101184-21systemd-run22:20:010
14649993821,13cyclictest12450irq/32-eth022:08:230
14649993821,13cyclictest12450irq/32-eth022:08:230
14649993811,18cyclictest959-21mta-sts-daemon22:13:280
14649993811,18cyclictest959-21mta-sts-daemon22:13:280
14651993730,4cyclictest0-21swapper/222:58:132
14651993729,5cyclictest23638-21idleruntime-cro19:40:012
14651993728,4cyclictest0-21swapper/222:33:252
1465099376,3cyclictest171rcu_preempt22:33:281
1465099373,27cyclictest2458627-21cups-browsed21:23:241
14650993731,3cyclictest0-21swapper/122:25:011
14650993731,3cyclictest0-21swapper/120:35:011
14650993730,3cyclictest0-21swapper/122:45:011
14650993726,8cyclictest32425-21timerwakeupswit20:08:351
14650993725,9cyclictest2458627-21cups-browsed21:38:271
14649993730,5cyclictest56526-21kernelversion21:18:200
14649993730,5cyclictest56526-21kernelversion21:18:200
14649993730,4cyclictest37132-21cron20:25:010
14649993730,4cyclictest135900-21ntp_kernel_pll_23:08:270
14649993728,7cyclictest195980-21cron00:35:010
14649993728,7cyclictest195980-21cron00:35:010
14649993724,10cyclictest38554-21unixbench_multi20:28:340
14651993630,3cyclictest0-21swapper/221:33:312
14651993628,5cyclictest64188-21proc_pri21:28:332
14651993628,4cyclictest0-21swapper/221:43:302
14651993627,6cyclictest29652-21sendmail20:00:012
14651993626,6cyclictest2458627-21cups-browsed21:39:222
14651993626,6cyclictest2458627-21cups-browsed21:17:422
14651993626,6cyclictest2458627-21cups-browsed21:17:422
1465199361,31cyclictest143264-21ssh23:18:322
14650993632,2cyclictest0-21swapper/123:38:241
14650993629,4cyclictest959-21mta-sts-daemon00:35:231
14650993628,5cyclictest50761-21latency_hist21:10:031
14649993630,4cyclictest20637-21munin-run19:30:010
14649993628,6cyclictest173388-21cups-browsed00:11:240
14649993627,6cyclictest50664-21systemd-run21:10:020
14649993622,5cyclictest151ktimers/022:58:240
14651993528,4cyclictest164361-21aten_r2power_vo23:48:192
14651993528,3cyclictest0-21swapper/222:48:242
14651993528,3cyclictest0-21swapper/222:48:242
14651993527,5cyclictest53451-21mailstats21:13:312
14651993527,5cyclictest53451-21mailstats21:13:312
14651993527,5cyclictest49134-21cron21:05:012
14651993527,5cyclictest22154-21systemd-run19:35:022
14651993527,5cyclictest1-21systemd20:40:022
1465099355,23cyclictest437-21mta-sts-daemon20:58:281
14650993527,5cyclictest64256-21mailstats21:28:331
14650993526,5cyclictest34021-21unixbench_multi20:13:371
14650993525,6cyclictest422-21dbus-daemon22:50:021
14650993522,9cyclictest422-21dbus-daemon23:00:011
14649993530,2cyclictest0-21swapper/020:40:020
14649993528,5cyclictest192113-21unixbench-2d00:28:370
14649993527,5cyclictest75901-21true21:45:020
14649993520,7cyclictest12450irq/32-eth023:33:260
14651993427,3cyclictest0-21swapper/222:23:192
14651993427,3cyclictest0-21swapper/200:30:022
14651993426,5cyclictest422-21dbus-daemon22:10:012
14651993426,5cyclictest422-21dbus-daemon22:10:012
14651993426,5cyclictest19689-21fschecks_time19:28:172
1465199342,26cyclictest185171-21ssh00:18:282
14651993421,8cyclictest168596-21sh23:53:312
14651993417,12cyclictest351ktimers/219:48:282
1465099343,23cyclictest178854-21ssh00:08:291
1465099343,23cyclictest178854-21ssh00:08:291
14650993426,4cyclictest0-21swapper/123:18:371
1465099342,5cyclictest171rcu_preempt19:18:231
14650993423,7cyclictest421-21avahi-daemon23:33:481
14650993417,13cyclictest422-21dbus-daemon22:48:231
14649993429,2cyclictest0-21swapper/021:50:010
14649993426,5cyclictest63832-21missed_timers21:28:220
14649993423,8cyclictest139893-21switchtime23:13:400
14649993423,4cyclictest0-21swapper/021:58:280
14649993421,10cyclictest103677-21proc_pri22:23:330
14649993410,20cyclictest114427-21ssh22:38:290
1465199333,23cyclictest181866-21ssh00:13:322
14651993325,5cyclictest59979-21fschecks_count21:23:162
14651993323,7cyclictest197952-21missed_timers00:38:212
14650993326,4cyclictest139161-21idleruntime23:13:221
14650993325,4cyclictest29575-21head19:58:351
14650993325,4cyclictest0-21swapper/100:23:241
1465099332,26cyclictest96209-21ssh22:13:231
1465099332,26cyclictest96209-21ssh22:13:231
14650993321,8cyclictest38053-21ntp_kernel_pll_20:28:241
1464999335,17cyclictest437-21mta-sts-daemon19:18:270
14649993327,4cyclictest132207-21needreboot23:03:210
14649993326,5cyclictest421-21avahi-daemon22:35:010
14649993310,12cyclictest151ktimers/019:23:240
1465199323,22cyclictest178741-21ssh00:08:252
1465199323,22cyclictest178741-21ssh00:08:252
14651993225,4cyclictest189553-21ls00:25:012
14651993224,5cyclictest47608-21cron21:00:012
14651993224,5cyclictest17633-21grep19:20:012
14651993223,4cyclictest0-21swapper/222:43:352
14650993225,5cyclictest0-21swapper/123:53:381
14650993224,5cyclictest26944-21aten_r2power_en19:53:131
14650993224,5cyclictest19137-21cron19:25:001
14650993223,6cyclictest1-21systemd20:03:291
14649993226,4cyclictest422-21dbus-daemon21:35:020
14649993226,4cyclictest172246-21sendmail_mailqu23:58:310
14649993226,4cyclictest172246-21sendmail_mailqu23:58:310
14649993222,4cyclictest49095-21head21:03:360
1464999322,23cyclictest185087-21awk00:18:250
14651993124,3cyclictest0-21swapper/223:28:292
14651993124,3cyclictest0-21swapper/220:25:012
14651993123,4cyclictest0-21swapper/200:33:372
14651993123,4cyclictest0-21swapper/200:33:372
1465199312,22cyclictest657-21snmpd00:03:252
14650993123,4cyclictest0-21swapper/122:03:261
14650993123,4cyclictest0-21swapper/122:03:261
14650993123,4cyclictest0-21swapper/121:48:151
14650993122,7cyclictest176757-21sh00:05:011
14650993121,7cyclictest0-21swapper/119:28:231
14649993123,5cyclictest421-21avahi-daemon21:21:030
1464999312,22cyclictest89193-21ntpq22:03:330
1464999312,22cyclictest89193-21ntpq22:03:330
14649993121,6cyclictest175765-21ssh00:03:290
1464999312,15cyclictest22370-21/usr/sbin/munin19:38:370
14649993117,11cyclictest0-21swapper/020:55:020
14649993117,10cyclictest0-21swapper/019:45:030
1465199303,20cyclictest657-21snmpd20:28:212
14651993022,5cyclictest453-21systemd-logind19:55:012
14651993022,5cyclictest2458627-21cups-browsed21:07:262
14651993020,7cyclictest34140-21systemd-run20:15:012
14650993024,3cyclictest179915-21kworker/u8:12-rpciod00:33:191
14650993024,3cyclictest179915-21kworker/u8:12-rpciod00:33:191
14650993024,3cyclictest0-21swapper/120:55:001
14649993022,5cyclictest0-21swapper/022:43:420
14649993021,6cyclictest433-21polkitd22:49:070
14649993021,6cyclictest433-21polkitd22:49:070
14651992921,5cyclictest46116-21cron20:55:002
14651992921,5cyclictest35627-21munin-run20:20:012
1464999297,15cyclictest151ktimers/020:33:230
14649992922,4cyclictest955-21mta-sts-daemon23:48:410
14649992921,6cyclictest31058-21unixbench_multi20:03:340
14651992821,4cyclictest15051-21telnet19:13:132
14651992819,6cyclictest266-21systemd-journal19:45:022
1465199281,22cyclictest89201-21sh22:03:332
1465199281,22cyclictest89201-21sh22:03:332
14650992820,6cyclictest133561-21sh23:05:011
14650992820,5cyclictest2458627-21cups-browsed20:48:431
14650992820,4cyclictest5158-21kworker/u8:11-xprtiod19:45:501
14649992822,3cyclictest0-21swapper/023:23:230
14649992820,5cyclictest2458627-21cups-browsed21:10:110
14649992820,5cyclictest2458627-21cups-browsed21:10:110
14649992810,14cyclictest151ktimers/019:33:240
14651992720,4cyclictest955-21mta-sts-daemon23:11:432
1465099276,5cyclictest0-21swapper/120:23:261
1465099272,17cyclictest657-21snmpd19:43:291
14650992721,5cyclictest271ktimers/119:33:231
14650992720,4cyclictest477-21rs:main1
1465099271,4cyclictest171rcu_preempt20:18:261
14649992719,6cyclictest34792-21kernelversion20:18:200
14651992619,3cyclictest0-21swapper/222:38:132
14651992618,5cyclictest421-21avahi-daemon20:48:322
14651992616,7cyclictest698-21cron20:45:012
14651992616,7cyclictest698-21cron20:45:012
14650992618,5cyclictest0-21swapper/120:45:011
14650992618,5cyclictest0-21swapper/120:45:011
14649992621,3cyclictest959-21mta-sts-daemon19:53:180
14649992620,4cyclictest29136-21open_inodes19:58:290
14649992619,4cyclictest124433-21head22:53:130
14649992617,7cyclictest16131-21munin-plugin-st19:15:010
1464999251,17cyclictest1994697-21sendmail-mta20:58:280
1465199245,15cyclictest437-21mta-sts-daemon20:33:272
14651992417,3cyclictest0-21swapper/220:03:132
14650992418,4cyclictest47652-21kworker/u8:14+xprtiod21:05:011
1464999241,18cyclictest657-21snmpd20:08:240
14651992314,6cyclictest32418-21sed20:08:352
14651992314,6cyclictest18498-21awk19:23:202
1464999232,15cyclictest42538-21ntpq20:43:220
1464999232,15cyclictest42538-21ntpq20:43:220
14649992318,2cyclictest0-21swapper/019:50:020
14649992317,4cyclictest44552-21tail20:48:370
14650992116,2cyclictest0-21swapper/119:10:021
14649992113,6cyclictest0-21swapper/019:10:020
14651991814,2cyclictest0-21swapper/219:10:022
1465299172,8cyclictest0-21swapper/322:13:253
1465299172,8cyclictest0-21swapper/322:13:253
1465299171,7cyclictest0-21swapper/321:46:023
1465299163,8cyclictest0-21swapper/322:38:293
1465299162,8cyclictest0-21swapper/323:03:273
1465299161,8cyclictest0-21swapper/322:23:263
1465299161,8cyclictest0-21swapper/320:38:293
1465299161,7cyclictest0-21swapper/320:28:233
1465299161,6cyclictest0-21swapper/321:03:223
1465299153,9cyclictest0-21swapper/322:53:263
1465299153,6cyclictest0-21swapper/321:13:293
1465299153,6cyclictest0-21swapper/321:13:293
1465299153,5cyclictest0-21swapper/322:33:283
1465299152,9cyclictest0-21swapper/323:53:233
1465299152,7cyclictest0-21swapper/322:03:203
1465299152,7cyclictest0-21swapper/322:03:203
1465299152,6cyclictest0-21swapper/319:48:253
1465299151,6cyclictest0-21swapper/319:28:223
1465299151,5cyclictest0-21swapper/323:08:333
1465299143,8cyclictest0-21swapper/321:10:023
1465299142,7cyclictest0-21swapper/323:58:303
1465299142,7cyclictest0-21swapper/323:58:303
1465299142,7cyclictest0-21swapper/322:48:253
1465299142,7cyclictest0-21swapper/322:48:253
1465299142,7cyclictest0-21swapper/322:18:343
1465299142,7cyclictest0-21swapper/321:23:243
1465299142,7cyclictest0-21swapper/319:15:023
1465299142,6cyclictest0-21swapper/321:58:223
1465299142,5cyclictest0-21swapper/320:18:303
1465299142,5cyclictest0-21swapper/319:23:243
1465299141,8cyclictest0-21swapper/323:33:223
1465299141,6cyclictest0-21swapper/323:48:203
1465299141,6cyclictest0-21swapper/320:13:253
1465299141,6cyclictest0-21swapper/319:38:243
1465299141,6cyclictest0-21swapper/319:33:253
1465299133,6cyclictest0-21swapper/321:18:173
1465299133,6cyclictest0-21swapper/321:18:173
1465299133,6cyclictest0-21swapper/300:03:243
1465299132,7cyclictest0-21swapper/323:38:243
1465299132,6cyclictest0-21swapper/300:28:293
1465299131,7cyclictest0-21swapper/323:18:273
1465299131,7cyclictest0-21swapper/322:28:303
1465299131,7cyclictest0-21swapper/322:08:233
1465299131,7cyclictest0-21swapper/322:08:233
1465299131,7cyclictest0-21swapper/321:28:183
1465299131,6cyclictest0-21swapper/319:58:243
1465299131,6cyclictest0-21swapper/319:53:213
1465299131,5cyclictest0-21swapper/320:08:233
1465299131,5cyclictest0-21swapper/319:18:223
1465299122,6cyclictest0-21swapper/300:21:023
1465299122,6cyclictest0-21swapper/300:13:213
1465299121,7cyclictest0-21swapper/323:13:193
1465299121,7cyclictest0-21swapper/321:33:363
1465299121,7cyclictest0-21swapper/300:18:253
1465299121,7cyclictest0-21swapper/300:08:333
1465299121,7cyclictest0-21swapper/300:08:333
1465299121,6cyclictest0-21swapper/322:58:203
1465299121,6cyclictest0-21swapper/322:43:183
1465299121,6cyclictest0-21swapper/320:58:283
1465299121,6cyclictest0-21swapper/320:03:233
1465299121,6cyclictest0-21swapper/300:38:273
1465299121,5cyclictest0-21swapper/323:28:293
1465299121,5cyclictest0-21swapper/319:43:253
1465299121,3cyclictest0-21swapper/320:33:233
1465299112,6cyclictest0-21swapper/321:45:013
1465299112,5cyclictest0-21swapper/323:23:183
1465299111,6cyclictest0-21swapper/321:53:203
1465299111,6cyclictest0-21swapper/320:53:193
1465299111,6cyclictest0-21swapper/320:23:163
1465299111,6cyclictest0-21swapper/300:33:153
1465299111,6cyclictest0-21swapper/300:33:153
1465299103,5cyclictest0-21swapper/323:43:283
1465299101,5cyclictest0-21swapper/321:38:343
146529991,5cyclictest0-21swapper/320:48:263
146529991,4cyclictest0-21swapper/320:43:233
146529991,4cyclictest0-21swapper/320:43:233
146529971,3cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional