You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 21:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Mon Nov 24, 2025 12:39:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3177080997964,10cyclictest1-21systemd09:45:021
3177081997862,11cyclictest3235888-40mandb09:45:252
3177079997453,14cyclictest3235936-21ssh09:44:590
3177079997153,15cyclictest3235888-40mandb09:45:250
3177080996852,7cyclictest0-21swapper/109:45:541
3177079996758,5cyclictest1-21systemd11:35:020
3177079996150,8cyclictest2319198-21cups-browsed07:40:010
317708099504,39cyclictest3284029-21ssh10:59:201
3177080995040,6cyclictest1-21systemd08:40:021
3177080994739,5cyclictest3181595-21systemd-run07:25:011
3177079994738,7cyclictest3338191-21unixbench_multi12:19:250
3177081994638,5cyclictest3346096-21sshd12:31:112
317707999463,33cyclictest151ktimers/010:19:200
3177081994538,4cyclictest3352293-21kworker/u8:8-xprtiod12:40:022
3177081994531,8cyclictest0-21swapper/209:45:042
3177080994536,6cyclictest3307730-21unixbench_singl11:34:241
3177079994531,11cyclictest3220012-21munin-plugin-st09:20:000
3177080994436,5cyclictest3202598-21cron08:35:011
3177080994335,5cyclictest3258467-21awk10:20:011
3177079994321,16cyclictest12450irq/32-eth011:09:130
3177080994232,6cyclictest3334069-21needreboot12:14:171
3177079994236,4cyclictest3321055-21munin-plugin-st11:55:010
3177079994236,4cyclictest3321055-21munin-plugin-st11:55:010
3177079994235,3cyclictest0-21swapper/010:29:170
3177079994231,8cyclictest3248672-21munin-plugin-st10:05:010
3177081994133,5cyclictest3204099-21idleruntime-cro08:40:012
3177079994135,3cyclictest0-21swapper/008:10:020
3177079994133,5cyclictest3304637-21latency_hist11:30:030
3177081994032,5cyclictest422-21dbus-daemon11:30:022
3177080994033,3cyclictest0-21swapper/111:25:011
3177080994032,5cyclictest491-21gdbus10:14:231
3177080994032,3cyclictest0-21swapper/111:19:161
317708199394,29cyclictest3294268-21ssh11:14:172
3177081993931,5cyclictest3324488-21cut12:00:002
3177081993931,5cyclictest3324488-21cut12:00:002
3177081993931,5cyclictest3297890-21systemd-run11:20:022
3177081993931,5cyclictest3288287-21latency_hist11:05:022
3177081993931,5cyclictest3284797-21cron11:00:012
3177081993931,5cyclictest3184566-21munin-run07:35:012
3177081993931,5cyclictest3184566-21munin-run07:35:012
317708099393,22cyclictest3216023-21ssh09:14:141
3177080993931,5cyclictest3280323-21kernelversion10:54:121
3177080993929,7cyclictest0-21swapper/111:10:011
3177081993828,6cyclictest3213113-21idleruntime-cro09:10:012
3177080993829,6cyclictest3244295-21kernelversion09:59:111
3177080993829,6cyclictest3226335-21true09:30:011
3177080993829,6cyclictest0-21swapper/109:35:021
3177080993828,6cyclictest491-21gdbus11:39:211
3177079993831,4cyclictest3216293-21sendmail_mailqu09:14:220
3177079993829,6cyclictest2319198-21cups-browsed10:23:500
3177079993821,11cyclictest12450irq/32-eth012:34:180
3177081993726,7cyclictest1-21systemd08:05:022
3177081993724,10cyclictest3213112-21diskmemload09:54:242
317708099374,26cyclictest3270731-21ssh10:39:151
3177080993730,3cyclictest0-21swapper/111:04:111
3177080993728,5cyclictest3241987-21unixbench_singl09:54:271
3177080993727,6cyclictest3195493-21aten_r2power_po08:14:031
3177079993727,7cyclictest3268293-21cron10:35:010
3177081993630,3cyclictest0-21swapper/209:15:012
3177080993628,5cyclictest1-21systemd11:30:021
317708099362,27cyclictest3263975-21ssh10:29:151
3177079993629,4cyclictest3297158-21proc_pri11:19:210
3177079993627,6cyclictest3329873-21/usr/sbin/munin12:09:020
3177079993624,9cyclictest3225188-21kernelversion09:29:100
317707999362,27cyclictest3241343-21ssh09:54:120
3177079993622,2cyclictest0-21swapper/010:39:110
3177079993621,10cyclictest151ktimers/008:19:120
3177081993528,4cyclictest3327656-21grep12:04:232
3177081993528,4cyclictest3317619-21kworker/u8:12+xprtiod11:50:022
3177081993528,3cyclictest0-21swapper/209:24:142
3177081993527,5cyclictest3344154-21fschecks_count12:29:082
3177080993529,3cyclictest0-21swapper/111:15:011
3177080993528,4cyclictest3316210-21aten_r2power_vo11:49:061
3177080993527,4cyclictest0-21swapper/112:19:251
3177080993526,6cyclictest3220016-21idleruntime-cro09:20:001
3177080993526,5cyclictest3211608-21cron09:05:001
3177079993529,3cyclictest3229634-21latency_hist09:35:020
3177079993528,4cyclictest3280499-21needreboot10:54:150
317708199345,18cyclictest437-21mta-sts-daemon07:39:172
3177081993429,2cyclictest0-21swapper/210:00:012
3177081993427,3cyclictest0-21swapper/211:39:192
3177081993426,5cyclictest956-21mta-sts-daemon07:59:102
3177081993426,5cyclictest3250402-21sh10:09:062
3177081993426,5cyclictest2319198-21cups-browsed08:29:412
3177081993423,7cyclictest2319198-21cups-browsed11:20:542
3177080993425,6cyclictest3205519-21unixbench_multi08:44:271
3177080993425,6cyclictest3205519-21unixbench_multi08:44:271
3177080993424,6cyclictest2319198-21cups-browsed10:44:231
3177080993419,9cyclictest3267374-21sh10:34:141
3177080993413,5cyclictest28-21ksoftirqd/112:36:231
3177080993412,5cyclictest28-21ksoftirqd/111:44:351
317707999343,26cyclictest3294309-21sh11:14:190
3177079993428,4cyclictest3345112-21unixbench_multi12:29:260
3177079993428,3cyclictest3202622-21systemd-run08:35:010
3177079993428,3cyclictest0-21swapper/011:59:190
3177079993428,3cyclictest0-21swapper/011:59:190
3177079993426,5cyclictest3244093-21cpuspeed09:59:060
3177079993425,7cyclictest3209516-21ntp_kernel_pll_08:59:150
3177079993416,12cyclictest12450irq/32-eth012:14:180
3177081993328,2cyclictest0-21swapper/210:49:262
3177081993327,3cyclictest0-21swapper/211:54:092
3177081993327,3cyclictest0-21swapper/211:54:092
3177081993327,3cyclictest0-21swapper/211:34:122
3177081993327,3cyclictest0-21swapper/210:20:022
3177081993327,3cyclictest0-21swapper/209:05:002
3177081993326,4cyclictest957-21mta-sts-daemon07:44:172
3177081993324,7cyclictest3280991-21timerandwakeup10:54:292
317708099339,6cyclictest28-21ksoftirqd/109:09:201
317708099333,25cyclictest3320312-21ssh11:54:201
317708099333,25cyclictest3320312-21ssh11:54:201
317708099333,24cyclictest3232028-21ssh09:39:181
3177080993326,3cyclictest0-21swapper/110:49:111
3177080993323,5cyclictest0-21swapper/110:04:271
3177080993314,4cyclictest28-21ksoftirqd/108:49:131
3177080993312,4cyclictest28-21ksoftirqd/107:59:121
3177080993311,5cyclictest28-21ksoftirqd/107:34:161
3177080993311,5cyclictest28-21ksoftirqd/107:34:161
3177079993327,4cyclictest3310349-21ssh11:39:080
3177079993325,5cyclictest3352285-21systemd-run12:40:010
3177079993321,8cyclictest0-21swapper/012:04:020
3177081993226,3cyclictest0-21swapper/212:20:012
3177081993225,3cyclictest0-21swapper/212:14:242
3177081993225,3cyclictest0-21swapper/209:35:112
3177081993224,4cyclictest0-21swapper/210:14:192
3177080993224,5cyclictest3348738-21sh12:35:011
3177080993224,5cyclictest3223055-21cut09:25:011
3177080993224,5cyclictest3187499-21unixbench_multi07:44:241
3177080993212,5cyclictest28-21ksoftirqd/112:07:031
3177080993212,10cyclictest28-21ksoftirqd/108:09:131
3177080993211,6cyclictest28-21ksoftirqd/112:22:301
3177080993211,5cyclictest28-21ksoftirqd/112:01:251
3177080993211,4cyclictest28-21ksoftirqd/107:29:171
3177080993210,4cyclictest28-21ksoftirqd/108:04:131
3177080993210,4cyclictest28-21ksoftirqd/107:49:201
3177080993210,11cyclictest28-21ksoftirqd/110:22:011
3177079993226,4cyclictest3231357-21aten_r2power_po09:39:050
3177079993225,5cyclictest3208591-21cron08:55:010
3177079993224,5cyclictest3340466-21/usr/sbin/munin12:24:080
3177079993224,5cyclictest3316027-21/usr/sbin/munin11:49:060
3177079993223,6cyclictest2319198-21cups-browsed10:40:430
317707999322,24cyclictest3300231-21rm11:24:130
317708199314,13cyclictest3213112-21diskmemload12:22:432
3177081993125,3cyclictest0-21swapper/210:45:012
3177081993125,3cyclictest0-21swapper/210:34:162
3177081993125,2cyclictest0-21swapper/211:44:382
3177081993124,4cyclictest3195107-21sh08:10:012
3177081993124,3cyclictest0-21swapper/207:14:042
3177081993123,5cyclictest3247392-21/usr/sbin/munin10:04:112
3177081993123,4cyclictest0-21swapper/211:09:272
3177081993122,6cyclictest491-21gdbus08:19:262
3177081993122,6cyclictest2319198-21cups-browsed07:54:402
3177080993120,8cyclictest3208599-21cron08:55:011
3177080993112,4cyclictest28-21ksoftirqd/110:06:261
317708099311,20cyclictest3323632-21ssh11:59:181
317708099311,20cyclictest3323632-21ssh11:59:181
3177080993111,5cyclictest28-21ksoftirqd/107:19:131
3177080993111,4cyclictest28-21ksoftirqd/108:24:181
3177080993110,5cyclictest28-21ksoftirqd/108:19:121
3177079993123,6cyclictest3283878-21ntp_kernel_pll_10:59:150
3177081993024,4cyclictest955-21mta-sts-daemon10:21:452
3177081993022,5cyclictest3270619-21needreboot10:39:112
3177081993022,5cyclictest2319198-21cups-browsed09:33:032
3177080993015,3cyclictest171rcu_preempt12:29:071
3177080993012,4cyclictest28-21ksoftirqd/107:54:181
3177080993010,5cyclictest28-21ksoftirqd/108:59:191
3177080993010,5cyclictest28-21ksoftirqd/108:29:201
317707999303,23cyclictest3254327-21ssh10:14:200
3177079993023,5cyclictest2319198-21cups-browsed08:45:430
317707999302,24cyclictest3313817-21ssh11:44:350
3177079993017,10cyclictest266-21systemd-journal09:00:040
3177079993010,15cyclictest657-21snmpd10:07:190
317708199293,22cyclictest3330592-21ntpq12:09:122
3177081992923,3cyclictest0-21swapper/209:29:172
3177081992920,6cyclictest3199646-21systemd-run08:25:022
3177080992910,4cyclictest28-21ksoftirqd/107:39:051
3177079992922,5cyclictest474-21in:imuxsock08:15:010
3177079992922,4cyclictest3205646-21latency_hist08:45:010
3177079992922,4cyclictest3205646-21latency_hist08:45:010
3177079992922,4cyclictest3200169-21fschecks_time08:29:070
3177079992919,7cyclictest266-21systemd-journal07:12:030
3177081992818,7cyclictest3179458-21ntp_kernel_pll_07:19:132
317707999283,7cyclictest3288213-21munin-run11:05:010
3177079992819,6cyclictest3187568-21cron07:45:010
3177081992722,2cyclictest0-21swapper/210:29:112
3177081992719,4cyclictest0-21swapper/209:19:122
3177081992717,7cyclictest3201999-21ntp_kernel_pll_08:34:212
317708099279,4cyclictest28-21ksoftirqd/107:14:081
3177079992722,3cyclictest3198605-21cpuspeed08:24:050
3177079992721,3cyclictest12550irq/33-eth007:29:150
3177079992720,5cyclictest2319198-21cups-browsed09:21:180
3177079992719,6cyclictest3203146-21fschecks_time08:39:070
3177081992619,4cyclictest3188115-21fschecks_time07:49:142
3177081992618,5cyclictest3206169-21gltestperf08:49:082
3177081992617,6cyclictest2319198-21cups-browsed07:26:582
3177080992619,3cyclictest3166125-21kworker/u8:2-xprtiod07:10:011
317707999264,9cyclictest3277158-21ntpq10:49:130
3177079992620,4cyclictest3188976-21unixbench_multi07:49:310
3177079992619,6cyclictest3177078-21cyclictest09:09:250
3177079992618,5cyclictest2319198-21cups-browsed07:54:070
3177079992616,8cyclictest0-21swapper/007:25:020
3177081992519,4cyclictest959-21mta-sts-daemon08:44:382
3177081992519,4cyclictest959-21mta-sts-daemon08:44:382
3177079992518,5cyclictest955-21mta-sts-daemon08:04:090
317708199242,17cyclictest3208010-21sshd08:54:112
3177081992417,5cyclictest3195986-21ld08:14:122
3177081992417,4cyclictest3208917-21telnet08:59:032
3177079992315,6cyclictest3183956-21ntp_kernel_pll_07:34:140
3177079992315,6cyclictest3183956-21ntp_kernel_pll_07:34:140
3177081992214,3cyclictest0-21swapper/207:23:102
3177079992214,5cyclictest3177085-21true07:10:010
3177079992212,2cyclictest0-21swapper/007:20:020
3177079992113,5cyclictest3191953-21unixbench_multi07:59:240
317708299203,9cyclictest0-21swapper/311:24:183
317708299202,10cyclictest0-21swapper/309:44:583
317708299193,9cyclictest0-21swapper/311:14:203
317708299183,7cyclictest0-21swapper/311:04:173
317708299182,8cyclictest0-21swapper/311:29:133
317708299182,10cyclictest0-21swapper/309:45:213
317708299181,9cyclictest0-21swapper/311:44:403
317708299173,8cyclictest0-21swapper/309:54:203
317708299172,9cyclictest0-21swapper/310:04:283
317708299172,8cyclictest0-21swapper/311:59:193
317708299172,8cyclictest0-21swapper/311:59:193
3177081991712,3cyclictest0-21swapper/207:10:032
317708299162,7cyclictest0-21swapper/311:09:163
317708299162,6cyclictest0-21swapper/307:19:133
317708299161,9cyclictest0-21swapper/310:59:123
317708299152,8cyclictest0-21swapper/311:39:143
317708299152,8cyclictest0-21swapper/309:34:173
317708299152,7cyclictest0-21swapper/312:34:133
317708299152,7cyclictest0-21swapper/312:29:143
317708299152,7cyclictest0-21swapper/309:29:193
317708299152,7cyclictest0-21swapper/309:19:143
317708299151,7cyclictest0-21swapper/309:09:173
317708299151,6cyclictest0-21swapper/307:14:153
317708299142,7cyclictest0-21swapper/312:19:193
317708299142,7cyclictest0-21swapper/311:19:213
317708299142,7cyclictest0-21swapper/310:49:173
317708299142,6cyclictest0-21swapper/308:49:213
317708299142,5cyclictest0-21swapper/308:34:193
317708299141,8cyclictest0-21swapper/312:14:193
317708299141,7cyclictest0-21swapper/309:04:233
317708299141,7cyclictest0-21swapper/307:44:153
317708299141,6cyclictest0-21swapper/312:39:143
317708299141,6cyclictest0-21swapper/308:59:143
317708299133,6cyclictest0-21swapper/311:34:143
317708299132,7cyclictest0-21swapper/310:19:163
317708299132,6cyclictest0-21swapper/311:54:083
317708299132,6cyclictest0-21swapper/311:54:083
317708299132,6cyclictest0-21swapper/310:34:093
317708299132,6cyclictest0-21swapper/309:24:143
317708299132,6cyclictest0-21swapper/309:14:143
317708299132,5cyclictest0-21swapper/308:09:113
317708299131,7cyclictest0-21swapper/310:44:133
317708299131,7cyclictest0-21swapper/309:59:133
317708299131,6cyclictest0-21swapper/312:09:153
317708299131,6cyclictest0-21swapper/308:54:113
317708299131,6cyclictest0-21swapper/308:04:193
317708299122,7cyclictest0-21swapper/307:24:153
317708299122,6cyclictest0-21swapper/310:29:203
317708299122,6cyclictest0-21swapper/310:14:163
317708299122,6cyclictest0-21swapper/309:35:113
317708299122,5cyclictest0-21swapper/308:19:143
317708299122,5cyclictest0-21swapper/307:34:133
317708299122,5cyclictest0-21swapper/307:34:133
317708299121,6cyclictest0-21swapper/312:04:113
317708299121,6cyclictest0-21swapper/310:54:103
317708299121,6cyclictest0-21swapper/310:05:243
317708299121,6cyclictest0-21swapper/308:39:123
317708299121,6cyclictest0-21swapper/307:59:133
317708299121,6cyclictest0-21swapper/307:54:163
317708299121,6cyclictest0-21swapper/307:39:123
317708299121,6cyclictest0-21swapper/307:29:163
317708299112,6cyclictest0-21swapper/308:24:113
317708299112,5cyclictest0-21swapper/310:39:053
317708299111,7cyclictest0-21swapper/311:49:233
317708299111,6cyclictest0-21swapper/312:24:193
317708299102,5cyclictest0-21swapper/310:24:093
317708299101,5cyclictest0-21swapper/308:44:153
317708299101,5cyclictest0-21swapper/308:44:153
317708299101,5cyclictest0-21swapper/308:29:063
317708299101,5cyclictest0-21swapper/308:14:153
31770829991,5cyclictest0-21swapper/307:49:193
31770829983,4cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional