You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 14:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Fri Nov 28, 2025 12:39:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2819704996052,5cyclictest422-21dbus-daemon07:10:010
2819704996041,15cyclictest0-21swapper/012:34:190
2819704995945,6cyclictest2915122-21ssh10:39:190
2819704995945,6cyclictest2915122-21ssh10:39:190
2819706995041,5cyclictest422-21dbus-daemon11:50:012
281970599503,36cyclictest2868765-21ssh09:29:151
2819704995023,16cyclictest12450irq/32-eth012:04:150
2819706994939,6cyclictest2905809-21basename10:25:012
2819706994933,12cyclictest2915875-21awk10:40:012
2819706994933,12cyclictest2915875-21awk10:40:012
2819704994936,7cyclictest0-21swapper/008:34:300
2819704994936,7cyclictest0-21swapper/008:34:300
2819704994931,5cyclictest2894651-21kernelversion10:09:110
2819705994638,5cyclictest2902501-21systemd-run10:20:011
2819705994638,5cyclictest2902501-21systemd-run10:20:011
2819704994639,4cyclictest0-21swapper/009:04:280
281970699450,4cyclictest171rcu_preempt09:34:192
2819705994539,3cyclictest0-21swapper/112:19:181
2819705994536,5cyclictest422-21dbus-daemon10:35:011
2819705994536,5cyclictest422-21dbus-daemon10:35:011
2819704994527,14cyclictest0-21swapper/010:50:000
2819704994527,14cyclictest0-21swapper/010:50:000
2819705994436,4cyclictest2839221-21latency_hist08:15:021
2819705994427,11cyclictest261rcuc/107:49:151
281970499445,31cyclictest2875409-21ssh09:39:230
2819704994436,5cyclictest422-21dbus-daemon09:35:010
2819706994333,6cyclictest1985041-21cups-browsed12:06:162
2819706994333,5cyclictest0-21swapper/211:34:172
281970699432,27cyclictest2855686-21diskmemload10:49:172
281970699432,27cyclictest2855686-21diskmemload10:49:172
281970599434,33cyclictest2976972-21ssh12:14:141
2819705994335,5cyclictest2944387-21munin-plugin-st11:25:011
281970599433,30cyclictest2904956-21ssh10:24:171
2819705994332,7cyclictest1-21systemd12:35:011
2819706994236,2cyclictest0-21swapper/211:45:002
2819706994236,2cyclictest0-21swapper/211:45:002
2819706994232,6cyclictest1985041-21cups-browsed11:55:012
2819705994233,6cyclictest2932341-21systemd-run11:05:021
2819704994232,7cyclictest2851183-21/usr/sbin/munin08:55:020
2819706994133,5cyclictest2895850-21systemd-run10:10:022
2819706994132,5cyclictest0-21swapper/208:59:192
2819706994128,9cyclictest0-21swapper/212:39:232
281970699411,36cyclictest2908407-21ssh10:29:182
281970699411,36cyclictest2908407-21ssh10:29:182
2819705994135,3cyclictest0-21swapper/112:00:011
2819705994132,4cyclictest0-21swapper/109:49:181
2819705994130,8cyclictest698-21cron09:45:011
2819705994130,8cyclictest422-21dbus-daemon11:00:011
2819704994134,3cyclictest0-21swapper/011:49:260
2819706994033,4cyclictest2830198-21grep07:45:012
2819706994031,6cyclictest2855717-21unin-run09:10:012
2819705994033,3cyclictest0-21swapper/109:40:011
2819705994029,7cyclictest421-21avahi-daemon12:05:211
2819704994031,5cyclictest0-21swapper/010:14:200
2819704994031,5cyclictest0-21swapper/010:14:200
2819704994019,13cyclictest12450irq/32-eth009:14:140
281970499400,35cyclictest2977076-21ssh12:14:180
2819706993931,5cyclictest2876178-21ssh09:40:022
2819704993924,12cyclictest2846622-21unixbench_singl08:39:270
2819706993830,5cyclictest2967653-21ssh12:00:012
2819706993827,7cyclictest2889088-21cron10:00:012
281970599383,26cyclictest2950255-21ssh11:34:171
281970599383,26cyclictest2946785-21ssh11:29:171
2819705993832,3cyclictest0-21swapper/109:55:021
2819705993827,8cyclictest2921859-21switchtime10:49:291
2819705993827,8cyclictest2921859-21switchtime10:49:291
2819704993831,5cyclictest2882882-21kworker/u8:4+xprtiod10:34:140
2819704993831,5cyclictest2882882-21kworker/u8:4+xprtiod10:34:140
2819704993831,3cyclictest0-21swapper/010:04:170
2819704993829,6cyclictest959-21mta-sts-daemon10:55:010
2819704993829,6cyclictest422-21dbus-daemon11:30:020
2819706993731,3cyclictest0-21swapper/207:10:012
2819706993727,8cyclictest453-21systemd-logind09:55:022
2819705993731,3cyclictest2954509-21kworker/u8:11-xprtiod11:45:001
2819705993731,3cyclictest2954509-21kworker/u8:11-xprtiod11:45:001
2819705993730,4cyclictest2915576-21tail10:39:251
2819705993730,4cyclictest2915576-21tail10:39:251
2819705993728,6cyclictest1985041-21cups-browsed10:14:301
2819705993728,6cyclictest1985041-21cups-browsed10:14:301
2819704993731,3cyclictest0-21swapper/011:14:210
2819704993727,7cyclictest266-21systemd-journal11:00:010
281970699364,24cyclictest2898284-21ssh10:14:162
281970699364,24cyclictest2898284-21ssh10:14:162
2819706993628,5cyclictest422-21dbus-daemon12:34:072
2819706993625,8cyclictest2869273-21timerwakeupswit09:29:262
2819706993625,4cyclictest0-21swapper/209:44:212
2819705993628,5cyclictest422-21dbus-daemon12:05:011
2819705993628,5cyclictest2822732-21systemd-run07:20:011
2819705993628,5cyclictest0-21swapper/109:20:011
2819705993627,6cyclictest266-21systemd-journal07:57:191
2819705993627,6cyclictest0-21swapper/111:20:011
281970499363,24cyclictest2941137-21sshd11:19:220
2819704993622,9cyclictest2918246-21needreboot10:44:190
2819704993622,9cyclictest2918246-21needreboot10:44:190
281970499361,28cyclictest2878536-21ssh09:44:160
2819706993530,2cyclictest0-21swapper/207:55:012
2819706993527,5cyclictest2828688-21munin-plugin-st07:40:002
2819705993529,3cyclictest0-21swapper/108:50:021
2819705993527,5cyclictest2983669-21latency12:24:121
2819705993527,5cyclictest2953787-21proc_pri11:39:241
281970599351,25cyclictest2963447-21ssh11:54:201
2819704993526,7cyclictest2821201-21systemd-run07:15:010
2819706993428,3cyclictest0-21swapper/212:15:012
2819706993427,3cyclictest0-21swapper/208:15:022
2819706993426,5cyclictest2941722-21cron11:20:012
2819705993428,3cyclictest0-21swapper/112:28:041
2819705993426,5cyclictest2924624-21latency10:54:191
2819705993426,5cyclictest2919097-21systemd-run10:45:011
2819705993426,5cyclictest2919097-21systemd-run10:45:011
2819705993424,4cyclictest0-21swapper/108:37:201
281970599341,25cyclictest2845207-21kworker/u8:5+rpciod09:14:161
2819704993428,3cyclictest0-21swapper/007:40:000
2819706993326,4cyclictest2842207-21debian-sa108:25:012
2819706993326,4cyclictest0-21swapper/210:45:002
2819706993326,4cyclictest0-21swapper/210:45:002
2819706993326,3cyclictest0-21swapper/207:19:192
2819706993324,7cyclictest2925606-21/usr/sbin/munin10:55:012
2819705993327,3cyclictest0-21swapper/107:55:021
2819705993325,5cyclictest2821335-21latency_hist07:15:021
2819705993325,5cyclictest1-21systemd10:10:021
2819705993323,7cyclictest2938095-21ntp_kernel_pll_11:14:161
2819705993322,8cyclictest2826637-21grep07:34:191
281970599332,22cyclictest2908485-21ssh10:29:211
281970599332,22cyclictest2908485-21ssh10:29:211
281970599331,27cyclictest2888199-21ssh09:59:181
2819704993325,5cyclictest2855638-21head09:09:290
2819704993325,4cyclictest0-21swapper/009:24:140
2819704993324,6cyclictest2834771-21latency_hist08:00:020
2819704993322,8cyclictest266-21systemd-journal09:20:010
281970699321,5cyclictest171rcu_preempt08:09:172
2819705993225,5cyclictest0-21swapper/108:20:011
2819705993225,4cyclictest2871614-21cut09:34:091
2819704993224,5cyclictest565-21gmain08:10:020
2819704993224,5cyclictest2842393-21latency_hist08:25:020
2819704993224,5cyclictest2824225-21systemd-run07:25:010
2819704993224,5cyclictest266-21systemd-journal11:35:030
2819704993224,5cyclictest151ktimers/012:29:160
2819704993221,5cyclictest12550irq/33-eth012:09:230
2819706993125,3cyclictest0-21swapper/212:04:262
2819706993123,5cyclictest1985041-21cups-browsed09:46:032
2819706993122,6cyclictest2935721-21latency_hist11:10:022
2819706993122,6cyclictest2865928-21unixbench-2d09:24:272
2819706993122,6cyclictest2840728-21systemd-run08:20:012
281970699311,19cyclictest2855686-21diskmemload10:34:192
281970699311,19cyclictest2855686-21diskmemload10:34:192
2819705993125,3cyclictest0-21swapper/108:34:361
2819705993125,3cyclictest0-21swapper/108:34:361
2819705993124,4cyclictest2994453-21munin-run12:40:011
2819705993123,4cyclictest0-21swapper/111:09:181
2819705993122,6cyclictest421-21avahi-daemon08:59:081
2819704993127,2cyclictest0-21swapper/012:20:010
2819704993124,4cyclictest2957844-21cron11:45:000
2819704993124,4cyclictest2957844-21cron11:45:000
2819704993124,4cyclictest2944080-21unixbench-2d11:24:270
2819704993122,5cyclictest2994437-21cron12:40:010
2819704993120,7cyclictest151ktimers/007:54:150
2819704993113,9cyclictest591-21sshd11:39:210
2819704993113,12cyclictest2868843-21ssh09:29:180
2819706993024,3cyclictest0-21swapper/212:20:012
2819706993024,3cyclictest0-21swapper/211:39:292
2819706993024,2cyclictest0-21swapper/210:59:242
2819706993023,4cyclictest959-21mta-sts-daemon10:17:562
2819706993023,4cyclictest959-21mta-sts-daemon10:17:562
2819706993023,4cyclictest2984060-21proc_pri12:24:232
2819706993023,3cyclictest0-21swapper/212:29:172
2819706993023,3cyclictest0-21swapper/211:24:092
2819706993022,5cyclictest2836089-21tail08:04:282
281970699301,25cyclictest2938066-21ssh11:14:152
2819705993023,4cyclictest0-21swapper/110:04:551
2819705993022,5cyclictest2842737-21fschecks_count08:29:071
2819705993022,5cyclictest2824753-21fschecks_time07:29:081
2819705993022,4cyclictest0-21swapper/111:45:321
2819704993024,3cyclictest0-21swapper/011:54:210
2819704993022,5cyclictest0-21swapper/007:19:170
281970499302,24cyclictest2881718-21ssh09:49:170
2819704993022,4cyclictest0-21swapper/007:34:190
2819704993020,7cyclictest2931802-21switchtime11:04:300
2819704993017,9cyclictest2904984-21ssh10:24:180
2819706992919,8cyclictest2831026-21needreboot07:49:142
2819705992923,3cyclictest0-21swapper/107:44:231
2819705992921,5cyclictest2847234-21cut08:44:081
2819705992919,8cyclictest2855054-21ntp_kernel_err09:09:161
2819705992919,7cyclictest1985038-21cupsd09:20:311
2819704992923,4cyclictest421-21avahi-daemon08:19:040
2819704992921,5cyclictest2902493-21systemd-run10:20:010
2819704992921,5cyclictest2902493-21systemd-run10:20:010
281970499291,23cyclictest2966837-21ntpq11:59:170
281970699283,19cyclictest2931505-21sshd11:04:212
2819706992821,4cyclictest2946378-21grep11:29:102
2819706992821,4cyclictest0-21swapper/209:19:252
2819706992819,6cyclictest955-21mta-sts-daemon08:54:172
2819706992819,6cyclictest2846583-21unixbench_multi08:39:272
2819706992817,8cyclictest2823536-21needreboot07:24:142
2819705992821,4cyclictest433-21polkitd08:04:291
2819704992822,3cyclictest0-21swapper/008:44:230
2819704992821,4cyclictest2984741-21cron12:25:010
2819704992818,7cyclictest421-21avahi-daemon11:08:340
281970699279,5cyclictest59950lldpd07:55:152
2819706992720,4cyclictest2825522-21timerwakeupswit07:29:262
2819706992719,4cyclictest0-21swapper/208:44:192
2819705992718,6cyclictest1-21systemd07:40:001
2819705992716,8cyclictest422-21dbus-daemon07:24:151
2819705992716,8cyclictest2841797-21mailstats08:24:231
281970499272,20cyclictest2869656-21kworker/u8:12+rpciod09:54:210
2819704992720,5cyclictest1985041-21cups-browsed08:49:070
2819704992720,5cyclictest0-21swapper/009:59:060
281970699263,16cyclictest2891582-21ssh10:04:182
2819706992620,3cyclictest2845434-21kworker/u8:8-xprtiod09:04:162
2819706992617,6cyclictest657-21snmpd09:14:022
2819705992621,3cyclictest2845207-21kworker/u8:5-xprtiod09:04:371
281970499262,18cyclictest2908387-21ntpq10:29:180
281970499262,18cyclictest2908387-21ntpq10:29:180
2819704992614,9cyclictest0-21swapper/007:30:010
2819704992611,9cyclictest12450irq/32-eth007:49:150
2819706992518,4cyclictest0-21swapper/208:50:012
2819705992518,4cyclictest2813215-21kworker/u8:8-xprtiod08:09:161
2819705992518,4cyclictest1985041-21cups-browsed08:54:161
2819704992519,4cyclictest2829052-21aten_r2power_fr07:44:070
2819706992418,3cyclictest2818496-21kworker/u8:12+xprtiod07:14:102
2819706992416,6cyclictest2845156-21unixbench_singl08:34:362
2819706992416,6cyclictest2845156-21unixbench_singl08:34:362
2819706992416,5cyclictest2843700-21idleruntime-cro08:30:002
2819706992415,6cyclictest474-21in:imuxsock07:34:202
2819704992418,4cyclictest2839205-21systemd-run08:15:010
281970499234,13cyclictest437-21mta-sts-daemon08:59:170
2819704992216,3cyclictest2843768-21latency_hist08:30:020
2819704992213,6cyclictest0-21swapper/008:04:110
2819705992012,5cyclictest2819721-21latency_hist07:10:021
281970799181,12cyclictest0-21swapper/310:24:203
281970799181,12cyclictest0-21swapper/308:54:153
281970799171,12cyclictest0-21swapper/311:59:153
281970799171,12cyclictest0-21swapper/309:14:133
281970799171,12cyclictest0-21swapper/307:49:203
281970799171,11cyclictest0-21swapper/311:44:183
281970799171,11cyclictest0-21swapper/311:44:183
281970799171,11cyclictest0-21swapper/309:19:203
281970799170,14cyclictest0-21swapper/307:10:013
281970799170,12cyclictest0-21swapper/310:14:223
281970799170,12cyclictest0-21swapper/310:14:223
281970799162,10cyclictest0-21swapper/311:14:153
281970799161,11cyclictest0-21swapper/312:39:203
281970799161,11cyclictest0-21swapper/312:34:223
281970799161,11cyclictest0-21swapper/312:29:133
281970799161,11cyclictest0-21swapper/312:24:153
281970799161,11cyclictest0-21swapper/312:14:193
281970799161,11cyclictest0-21swapper/312:04:163
281970799161,11cyclictest0-21swapper/311:54:203
281970799161,11cyclictest0-21swapper/311:49:233
281970799161,11cyclictest0-21swapper/311:39:183
281970799161,11cyclictest0-21swapper/311:34:143
281970799161,11cyclictest0-21swapper/311:29:163
281970799161,11cyclictest0-21swapper/310:59:163
281970799161,11cyclictest0-21swapper/310:54:223
281970799161,11cyclictest0-21swapper/310:49:183
281970799161,11cyclictest0-21swapper/310:49:183
281970799161,11cyclictest0-21swapper/310:44:243
281970799161,11cyclictest0-21swapper/310:44:243
281970799161,11cyclictest0-21swapper/310:39:123
281970799161,11cyclictest0-21swapper/310:39:123
281970799161,11cyclictest0-21swapper/309:34:163
281970799161,11cyclictest0-21swapper/309:29:143
281970799161,11cyclictest0-21swapper/309:24:153
281970799161,11cyclictest0-21swapper/309:09:203
281970799161,11cyclictest0-21swapper/308:44:153
281970799161,11cyclictest0-21swapper/308:39:153
281970799161,11cyclictest0-21swapper/308:24:203
281970799161,11cyclictest0-21swapper/308:19:163
281970799161,11cyclictest0-21swapper/308:09:163
281970799161,11cyclictest0-21swapper/307:59:173
281970799161,11cyclictest0-21swapper/307:54:183
281970799161,11cyclictest0-21swapper/307:29:133
281970799161,11cyclictest0-21swapper/307:14:183
281970799160,13cyclictest0-21swapper/311:19:153
281970799160,12cyclictest0-21swapper/312:19:153
281970799160,12cyclictest0-21swapper/310:34:193
281970799160,12cyclictest0-21swapper/310:34:193
281970799160,12cyclictest0-21swapper/308:14:223
281970799160,11cyclictest0-21swapper/311:24:143
281970799151,11cyclictest0-21swapper/312:09:183
281970799151,11cyclictest0-21swapper/311:04:183
281970799151,11cyclictest0-21swapper/310:27:213
281970799151,11cyclictest0-21swapper/310:27:213
281970799151,11cyclictest0-21swapper/310:19:203
281970799151,11cyclictest0-21swapper/310:19:203
281970799151,11cyclictest0-21swapper/310:09:203
281970799151,11cyclictest0-21swapper/310:04:163
281970799151,11cyclictest0-21swapper/307:19:193
281970799151,10cyclictest0-21swapper/311:05:433
281970799151,10cyclictest0-21swapper/309:42:123
281970799151,10cyclictest0-21swapper/309:04:243
281970799151,10cyclictest0-21swapper/308:59:133
281970799150,11cyclictest0-21swapper/309:56:263
281970799150,11cyclictest0-21swapper/309:49:193
281970799150,11cyclictest0-21swapper/308:34:243
281970799150,11cyclictest0-21swapper/308:34:243
281970799150,11cyclictest0-21swapper/308:29:173
281970799150,11cyclictest0-21swapper/308:04:213
281970799150,11cyclictest0-21swapper/307:44:153
281970799150,11cyclictest0-21swapper/307:39:143
281970799150,11cyclictest0-21swapper/307:24:193
281970799141,10cyclictest0-21swapper/309:35:213
281970799141,10cyclictest0-21swapper/308:46:203
281970799141,10cyclictest0-21swapper/307:35:023
281970799140,10cyclictest0-21swapper/309:50:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional