You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 14:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Sun Nov 23, 2025 12:39:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1100756996150,5cyclictest1108509-21kworker/u8:4-rpciod07:55:441
1100757995546,5cyclictest1211603-21idleruntime-cro11:00:002
1100757995546,5cyclictest1211603-21idleruntime-cro11:00:002
1100755995321,24cyclictest1217565-21ssh11:09:410
1100757995242,7cyclictest263320-21cups-browsed12:39:202
1100756995239,10cyclictest263320-21cups-browsed07:10:011
1100757995143,5cyclictest1204710-21systemd-run10:50:012
1100756995040,6cyclictest0-21swapper/109:25:011
1100755994934,12cyclictest1153379-21unixbench-2d09:34:250
1100757994839,6cyclictest1140491-21systemd-run09:15:022
1100755994639,4cyclictest1163221-21systemd-run09:50:010
1100757994536,6cyclictest1197656-21systemd-run10:40:012
1100757994536,6cyclictest1197656-21systemd-run10:40:012
1100756994539,3cyclictest0-21swapper/108:40:011
1100755994521,15cyclictest151ktimers/011:49:170
110075799434,9cyclictest1270036-21ssh12:24:112
110075799434,9cyclictest1270036-21ssh12:24:112
1100757994329,11cyclictest266-21systemd-journal11:05:012
1100757994329,11cyclictest266-21systemd-journal11:05:012
1100756994335,5cyclictest1123323-21systemd-run08:25:011
1100756994334,5cyclictest422-21dbus-daemon09:00:021
1100755994336,4cyclictest1156763-21cron09:40:010
110075599432,31cyclictest1220979-21ssh11:14:190
1100757994234,4cyclictest0-21swapper/209:25:022
1100757994233,6cyclictest1262008-21basename12:14:032
1100756994235,3cyclictest0-21swapper/110:04:171
1100756994234,5cyclictest1166352-21ls09:55:011
1100756994233,4cyclictest0-21swapper/110:34:111
1100756994233,4cyclictest0-21swapper/110:34:111
110075599423,31cyclictest1175841-21ssh10:09:160
110075599421,13cyclictest1225359-21kworker/u8:8+flush-0:4011:24:150
1100757994133,4cyclictest1129311-21kworker/u8:14+rpciod10:15:022
1100757994131,6cyclictest263320-21cups-browsed09:47:452
1100757994131,6cyclictest1-21systemd10:30:012
1100757994130,6cyclictest1221278-21cat11:14:252
1100756994133,5cyclictest1235308-21unixbench-2d11:34:251
1100755994134,4cyclictest1193133-21missed_timers10:34:090
1100755994134,4cyclictest1193133-21missed_timers10:34:090
1100757994030,7cyclictest422-21dbus-daemon11:20:012
1100756994033,4cyclictest1156769-21latency_hist09:40:011
1100756994031,5cyclictest1248218-21aten_r2power_en11:54:031
1100756994030,6cyclictest1130805-21systemd-run08:50:011
1100757993932,5cyclictest0-21swapper/209:54:082
1100757993931,4cyclictest959-21mta-sts-daemon11:30:582
1100757993928,7cyclictest1156990-21latency_hist09:40:032
110075799392,32cyclictest1182772-21ssh10:19:162
1100756993932,3cyclictest0-21swapper/111:10:011
1100756993931,5cyclictest1267397-21systemd-run12:20:021
1100756993931,5cyclictest1253097-21cron12:00:011
1100756993930,5cyclictest1136796-21idleruntime-cro09:10:011
1100755993929,7cyclictest1263894-21latency_hist12:15:030
1100755993923,12cyclictest1119329-21fschecks_count08:14:060
1100755993923,12cyclictest1119329-21fschecks_count08:14:060
1100755993911,24cyclictest1224327-21ssh11:19:150
1100757993830,5cyclictest957-21mta-sts-daemon10:34:192
1100757993830,5cyclictest957-21mta-sts-daemon10:34:192
1100757993830,5cyclictest1274492-21unixbench_singl12:29:232
1100757993830,5cyclictest1274492-21unixbench_singl12:29:232
1100757993830,5cyclictest1136912-21latency_hist09:10:022
1100757993829,5cyclictest1129283-21cron08:45:012
1100757993827,7cyclictest421-21avahi-daemon12:33:022
1100756993830,5cyclictest1169869-21munin-run10:00:011
1100757993727,6cyclictest1-21systemd10:25:022
1100757993727,6cyclictest1126280-21latency_hist08:35:012
1100757993726,7cyclictest263320-21cups-browsed09:28:462
1100757993725,8cyclictest1160019-21systemd-run09:45:022
110075799372,25cyclictest1266502-21ssh12:19:152
1100756993730,4cyclictest0-21swapper/112:25:031
1100756993730,4cyclictest0-21swapper/112:25:031
1100756993729,5cyclictest1228719-21latency_hist11:25:021
1100756993729,5cyclictest1211594-21idleruntime-cro11:00:001
1100756993729,5cyclictest1211594-21idleruntime-cro11:00:001
1100755993730,4cyclictest1214408-21mailstats11:04:240
1100755993730,4cyclictest1214408-21mailstats11:04:240
1100757993628,5cyclictest1105294-21cron07:25:012
1100757993625,7cyclictest1206614-21aten_r2power_en10:54:032
1100757993625,7cyclictest1206614-21aten_r2power_en10:54:032
1100757993619,12cyclictest351ktimers/211:24:162
110075699363,27cyclictest1189806-21ssh10:29:171
1100756993627,6cyclictest422-21dbus-daemon10:50:011
1100756993626,7cyclictest1119617-21needreboot08:14:101
1100756993626,7cyclictest1119617-21needreboot08:14:101
1100755993626,7cyclictest1210471-21latency10:59:100
1100755993626,7cyclictest1210471-21latency10:59:100
1100755993626,6cyclictest0-21swapper/009:10:010
1100757993528,4cyclictest956-21mta-sts-daemon11:07:272
1100757993527,5cyclictest421-21avahi-daemon08:19:102
1100757993527,5cyclictest421-21avahi-daemon08:19:102
1100757993527,5cyclictest263320-21cups-browsed11:42:582
1100757993527,4cyclictest0-21swapper/207:40:012
1100757993526,6cyclictest1118813-21systemd-run08:10:012
1100757993525,5cyclictest0-21swapper/211:56:052
1100757993524,7cyclictest421-21avahi-daemon10:41:022
1100757993524,7cyclictest421-21avahi-daemon10:41:022
1100756993527,5cyclictest1120837-21fschecks_time08:19:081
1100756993527,5cyclictest1120837-21fschecks_time08:19:081
1100756993527,3cyclictest0-21swapper/112:34:161
1100756993527,3cyclictest0-21swapper/110:20:021
110075699352,27cyclictest1224256-21ssh11:19:121
1100755993528,5cyclictest1242823-21munin-plugin-st11:45:010
110075599351,28cyclictest151ktimers/007:49:140
110075799343,26cyclictest1142934-21sh09:19:172
1100757993425,7cyclictest1244294-21sshd11:48:032
1100757993424,5cyclictest1108287-21sh07:35:002
1100757993422,9cyclictest1116664-21ntp_kernel_err08:04:102
1100757993422,6cyclictest0-21swapper/211:54:242
1100756993429,2cyclictest0-21swapper/110:15:011
1100756993426,5cyclictest1113335-21fschecks_time07:54:061
1100756993425,5cyclictest1150962-21sshd09:30:591
110075599345,25cyclictest1266457-21ssh12:19:130
1100755993427,4cyclictest151ktimers/012:04:150
1100755993427,4cyclictest1249762-21cron11:55:010
1100755993423,8cyclictest0-21swapper/009:41:110
110075799333,22cyclictest1259091-21ssh12:09:142
110075799333,22cyclictest1172501-21ssh10:04:132
1100757993326,4cyclictest956-21mta-sts-daemon07:29:132
1100757993326,4cyclictest956-21mta-sts-daemon07:29:132
1100757993325,5cyclictest1153465-21unixbench_singl09:34:252
1100757993324,6cyclictest263320-21cups-browsed12:01:462
1100757993323,6cyclictest263314-21cupsd11:29:412
1100756993327,3cyclictest0-21swapper/110:40:011
1100756993327,3cyclictest0-21swapper/110:40:011
1100756993326,4cyclictest0-21swapper/110:54:131
1100756993326,4cyclictest0-21swapper/110:54:131
1100756993324,6cyclictest263320-21cups-browsed11:14:151
1100755993326,5cyclictest12450irq/32-eth011:59:130
1100755993326,5cyclictest0-21swapper/010:20:010
1100755993326,4cyclictest1169869-21munin-run10:00:010
1100755993325,5cyclictest1123301-21munin-run08:25:000
1100755993324,6cyclictest1103796-21idleruntime-cro07:20:010
1100755993320,9cyclictest12450irq/32-eth010:04:140
110075599331,24cyclictest1179114-21ssh10:14:160
110075799324,18cyclictest955-21mta-sts-daemon08:39:152
1100757993222,7cyclictest1232415-21kworker/u8:5+xprtiod11:40:012
1100757993222,4cyclictest0-21swapper/207:14:202
110075799321,23cyclictest1168998-21ssh09:59:162
1100756993227,2cyclictest0-21swapper/109:20:021
1100756993225,4cyclictest0-21swapper/111:39:191
1100756993225,3cyclictest0-21swapper/108:34:171
1100756993224,5cyclictest1232181-21basename11:30:011
1100756993223,5cyclictest1140169-21unixbench_multi09:14:251
1100756993222,8cyclictest1260041-21sh12:10:021
1100756993222,7cyclictest474-21in:imuxsock09:50:011
1100756993222,7cyclictest1245993-21grep11:49:241
110075699322,26cyclictest263320-21cups-browsed12:35:031
1100755993226,4cyclictest956-21mta-sts-daemon09:28:490
1100755993226,4cyclictest1197662-21systemd-run10:40:010
1100755993226,4cyclictest1197662-21systemd-run10:40:010
1100755993226,3cyclictest0-21swapper/009:20:010
1100755993225,5cyclictest1258361-21aten_r2power_cu12:09:020
1100755993225,4cyclictest1208185-21kworker/u8:610:55:010
1100755993225,4cyclictest1208185-21kworker/u8:610:55:010
1100755993224,5cyclictest1109773-21cron07:40:010
1100755993222,7cyclictest956-21mta-sts-daemon10:42:200
1100755993222,7cyclictest956-21mta-sts-daemon10:42:200
1100755993222,7cyclictest1116348-21fschecks_time08:04:040
1100755993220,6cyclictest12450irq/32-eth011:34:190
1100757993122,5cyclictest0-21swapper/207:59:142
1100756993124,4cyclictest1119031-21kworker/u8:0-nfsiod09:43:521
1100756993124,3cyclictest0-21swapper/112:29:111
1100756993124,3cyclictest0-21swapper/112:29:111
1100756993122,6cyclictest1136789-21diskmemload10:10:011
1100756993122,5cyclictest1199743-21head10:44:031
1100756993122,5cyclictest1199743-21head10:44:031
110075699311,22cyclictest1262736-21ssh12:14:151
110075599315,3cyclictest171rcu_preempt10:24:150
1100755993126,2cyclictest0-21swapper/012:35:010
1100755993125,4cyclictest1190319-21grep10:29:260
1100755993124,3cyclictest0-21swapper/012:39:130
1100755993121,6cyclictest12450irq/32-eth007:55:460
1100755993120,9cyclictest1124667-21unixbench-2d08:29:230
1100757993022,4cyclictest955-21mta-sts-daemon07:54:112
1100757993021,4cyclictest0-21swapper/207:44:182
110075699305,19cyclictest1214137-21ssh11:04:151
110075699305,19cyclictest1214137-21ssh11:04:151
1100756993023,3cyclictest0-21swapper/110:24:171
1100756993022,4cyclictest0-21swapper/111:44:171
1100756993020,7cyclictest1103124-21needreboot07:19:101
1100755993024,4cyclictest1221788-21kworker/u8:2+xprtiod12:25:010
1100755993024,4cyclictest1221788-21kworker/u8:2+xprtiod12:25:010
1100755993022,5cyclictest263320-21cups-browsed07:29:130
1100755993022,5cyclictest263320-21cups-browsed07:29:130
110075799292,4cyclictest171rcu_preempt08:14:172
110075799292,4cyclictest171rcu_preempt08:14:172
1100757992920,5cyclictest959-21mta-sts-daemon08:25:022
1100757992920,5cyclictest1123832-21fschecks_count08:29:052
1100757992920,5cyclictest0-21swapper/210:10:012
1100756992922,4cyclictest1254934-21aten_r2power_po12:04:031
1100756992921,5cyclictest1132271-21cron08:55:011
1100756992921,4cyclictest0-21swapper/109:25:361
1100756992920,5cyclictest0-21swapper/108:45:011
110075599295,7cyclictest171rcu_preempt08:49:160
110075599294,13cyclictest437-21mta-sts-daemon08:19:150
110075599294,13cyclictest437-21mta-sts-daemon08:19:150
110075599293,21cyclictest1146349-21ssh09:24:180
1100755992923,3cyclictest0-21swapper/007:35:000
110075599291,24cyclictest1165037-21cut09:54:100
1100757992815,6cyclictest1124892-21kworker/u8:1-xprtiod09:00:012
1100755992822,3cyclictest0-21swapper/008:34:140
110075599282,20cyclictest1273908-21ssh12:29:130
110075599282,20cyclictest1273908-21ssh12:29:130
1100755992819,7cyclictest1230118-21ssh11:28:030
1100755992816,9cyclictest12450irq/32-eth009:14:190
1100757992721,3cyclictest0-21swapper/207:20:032
1100756992720,4cyclictest1118786-21cron08:10:011
1100756992719,5cyclictest1110346-21fschecks_time07:44:081
1100756992716,4cyclictest957-21mta-sts-daemon07:14:211
110075599271,21cyclictest1238336-21ssh11:39:180
1100757992618,5cyclictest1-21systemd08:55:012
1100756992620,3cyclictest0-21swapper/107:39:231
1100756992616,7cyclictest1111527-21sshd07:47:391
110075599262,19cyclictest1118340-21ps08:09:180
1100755992620,3cyclictest0-21swapper/007:45:010
1100755992611,13cyclictest1132272-21cron08:55:010
110075599260,18cyclictest1203169-21aten_r2power_fr10:49:010
110075799253,12cyclictest422-21dbus-daemon09:04:112
1100757992519,3cyclictest0-21swapper/208:50:012
1100757992516,6cyclictest1112194-21grep07:49:152
1100756992518,4cyclictest1107344-21fschecks_time07:34:261
1100756992518,4cyclictest1106702-21unixbench_multi07:29:221
1100756992518,4cyclictest1106702-21unixbench_multi07:29:221
1100756992517,5cyclictest1124695-21tail08:29:231
1100755992521,2cyclictest0-21swapper/007:25:010
1100755992518,5cyclictest266-21systemd-journal07:15:010
1100755992517,5cyclictest263320-21cups-browsed08:55:080
1100755992514,4cyclictest1135249-21unixbench_singl09:04:250
110075599251,18cyclictest657-21snmpd08:39:150
1100756992416,5cyclictest1135329-21latency_hist09:05:021
1100756992414,7cyclictest1104564-21missed_timers07:24:071
1100756992314,3cyclictest955-21mta-sts-daemon08:04:151
110075599232,15cyclictest1113755-21ntp_states07:54:170
1100755992117,3cyclictest0-21swapper/007:10:010
1100755992014,4cyclictest263320-21cups-browsed08:42:170
110075899191,9cyclictest0-21swapper/311:29:153
110075899162,8cyclictest0-21swapper/309:34:173
110075899161,8cyclictest0-21swapper/312:14:183
110075799161,11cyclictest1-21systemd07:10:012
110075899153,5cyclictest0-21swapper/310:29:143
110075899152,7cyclictest0-21swapper/311:34:203
110075899152,7cyclictest0-21swapper/310:04:133
110075899152,6cyclictest0-21swapper/309:09:153
110075899152,4cyclictest0-21swapper/308:24:143
110075899151,8cyclictest0-21swapper/312:29:093
110075899151,8cyclictest0-21swapper/312:29:093
110075899151,8cyclictest0-21swapper/310:44:113
110075899151,8cyclictest0-21swapper/310:44:113
110075899151,8cyclictest0-21swapper/309:54:243
110075899151,6cyclictest0-21swapper/310:24:123
110075899151,6cyclictest0-21swapper/307:29:093
110075899151,6cyclictest0-21swapper/307:29:093
110075899142,8cyclictest0-21swapper/307:55:443
110075899142,7cyclictest0-21swapper/311:14:193
110075899142,7cyclictest0-21swapper/310:54:133
110075899142,7cyclictest0-21swapper/310:54:133
110075899142,6cyclictest0-21swapper/309:14:153
110075899141,7cyclictest0-21swapper/311:59:113
110075899141,7cyclictest0-21swapper/310:19:203
110075899141,7cyclictest0-21swapper/309:24:163
110075899141,6cyclictest0-21swapper/311:39:153
110075899141,6cyclictest0-21swapper/308:09:123
110075899141,6cyclictest0-21swapper/308:04:173
110075899133,6cyclictest0-21swapper/309:49:153
110075899133,5cyclictest0-21swapper/312:39:093
110075899133,5cyclictest0-21swapper/312:09:193
110075899132,7cyclictest0-21swapper/312:24:123
110075899132,7cyclictest0-21swapper/312:24:123
110075899132,6cyclictest0-21swapper/311:24:173
110075899132,5cyclictest0-21swapper/308:54:153
110075899132,4cyclictest0-21swapper/307:44:133
110075899131,7cyclictest0-21swapper/310:14:113
110075899131,7cyclictest0-21swapper/309:19:143
110075899131,6cyclictest0-21swapper/311:54:153
110075899131,6cyclictest0-21swapper/310:49:013
110075899123,5cyclictest0-21swapper/312:19:193
110075899121,6cyclictest0-21swapper/312:04:033
110075899121,6cyclictest0-21swapper/311:09:403
110075899121,6cyclictest0-21swapper/310:39:123
110075899121,6cyclictest0-21swapper/310:39:123
110075899121,6cyclictest0-21swapper/308:49:163
110075899121,6cyclictest0-21swapper/308:19:143
110075899121,6cyclictest0-21swapper/308:19:143
110075899121,6cyclictest0-21swapper/307:34:393
110075899121,5cyclictest0-21swapper/307:49:133
110075899121,5cyclictest0-21swapper/307:15:363
110075899120,6cyclictest0-21swapper/308:44:113
110075899115,3cyclictest0-21swapper/309:29:183
110075899113,4cyclictest0-21swapper/307:40:013
110075899112,5cyclictest0-21swapper/310:09:113
110075899112,5cyclictest0-21swapper/309:59:073
110075899112,5cyclictest0-21swapper/308:14:123
110075899112,5cyclictest0-21swapper/308:14:123
110075899111,6cyclictest0-21swapper/312:33:033
110075899111,6cyclictest0-21swapper/310:59:083
110075899111,6cyclictest0-21swapper/310:59:083
110075899111,6cyclictest0-21swapper/310:34:143
110075899111,6cyclictest0-21swapper/310:34:143
110075899111,6cyclictest0-21swapper/309:44:063
110075899111,6cyclictest0-21swapper/309:38:533
110075899111,5cyclictest0-21swapper/308:39:133
110075899111,5cyclictest0-21swapper/308:32:573
110075899111,5cyclictest0-21swapper/307:14:183
110075899110,6cyclictest0-21swapper/308:29:173
110075899103,5cyclictest0-21swapper/308:59:253
110075899102,5cyclictest0-21swapper/311:46:043
110075899102,5cyclictest0-21swapper/311:44:033
110075899102,5cyclictest0-21swapper/311:04:073
110075899102,5cyclictest0-21swapper/311:04:073
110075899101,6cyclictest0-21swapper/307:54:153
110075899101,5cyclictest0-21swapper/311:18:023
110075899101,5cyclictest0-21swapper/307:24:123
11007589991,5cyclictest0-21swapper/309:04:063
11007589972,4cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional