You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-26 - 06:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Wed Aug 26, 2026 00:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1011688996660,4cyclictest1058488-21apt-get19:55:110
1011688994845,2cyclictest1229585-21kworker/u8:2+events_unbound22:45:240
1011698994541,3cyclictest1131334-21kworker/u8:2+flush-259:021:25:263
1011696994439,4cyclictest1162402-21kworker/u8:3+flush-259:021:35:262
1011696994439,4cyclictest1162402-21kworker/u8:3+flush-259:021:35:252
1011696994034,4cyclictest1151835-21latency_hist21:25:002
1011688993936,2cyclictest1038048-21kworker/u8:5+events_unbound20:40:250
1011698993834,3cyclictest1229585-21kworker/u8:2+events_unbound23:40:113
1011698993832,4cyclictest1261071-21apt-get23:10:113
1011692993832,4cyclictest1255893-21apt-get23:05:111
1011688993835,2cyclictest1318291-21kworker/u8:1+events_unbound00:20:220
1011688993731,4cyclictest1100234-21cp20:35:010
1011688993730,5cyclictest1318276-21latency_hist00:05:010
1011688993633,2cyclictest1229585-21kworker/u8:2+events_unbound23:35:150
1011688993633,2cyclictest1141476-21kworker/u8:1+events_unbound21:40:150
1011692993430,3cyclictest1038048-21kworker/u8:5+flush-259:019:45:001
1011692993421,4cyclictest1177740-21latency_hist21:50:001
1011688993431,2cyclictest1105304-21kworker/u8:0+events_unbound21:15:240
1011688993431,2cyclictest1105304-21kworker/u8:0+events_unbound21:00:230
1011698993330,2cyclictest1105304-21kworker/u8:0+flush-259:021:15:213
1011698993330,2cyclictest1001618-21kworker/u8:1+flush-259:019:40:193
1011692993328,4cyclictest1162402-21kworker/u8:3+flush-259:021:50:241
1011698993229,2cyclictest1188056-21kworker/u8:2+flush-259:022:25:163
1011696993226,4cyclictest1344452-21apt-get00:30:132
1011688993230,2cyclictest1131334-21kworker/u8:2+events_unbound21:20:260
1011688993230,2cyclictest1048104-21kworker/u8:3+events_unbound19:50:140
1011688993230,2cyclictest1047781-21kworker/u8:0+events_unbound20:00:140
1011698993128,2cyclictest1318291-21kworker/u8:1+events_unbound00:20:003
1011698993127,3cyclictest1063425-21kworker/u8:1+flush-259:020:35:213
1011692993127,3cyclictest1261036-21kworker/u8:4+events_unbound23:25:001
1011692993126,3cyclictest1099848-21base6420:35:011
1011688993130,1cyclictest1213848-21kworker/u8:0+events_unbound22:50:220
1011688993128,2cyclictest1047781-21kworker/u8:0+events_unbound20:05:150
1011698993027,2cyclictest1115126-21kworker/u8:3+events_unbound21:05:003
1011698993026,3cyclictest1219277-21/usr/sbin/munin22:30:153
1011696993025,4cyclictest1038048-21kworker/u8:5+flush-259:020:25:002
1011696993022,5cyclictest335557-21systemd-journal20:35:012
1011692993024,4cyclictest1078973-21latency_hist20:15:011
1011688993028,2cyclictest1172312-21kworker/u8:0+events_unbound21:50:280
1011688993027,2cyclictest1229585-21kworker/u8:2+events_unbound23:20:220
1011688993026,3cyclictest1152102-21apt-get21:25:120
1011688993026,3cyclictest1032562-21apt-get19:30:140
1011698992926,2cyclictest1105304-21kworker/u8:0+events_unbound20:45:593
1011698992922,5cyclictest1203932-21diskstats22:15:143
1011688992926,2cyclictest1318291-21kworker/u8:1+events_unbound00:30:240
1011688992926,2cyclictest1297528-21kworker/u8:0+events_unbound23:45:200
1011688992926,2cyclictest1297528-21kworker/u8:0+events_unbound23:45:200
1011688992926,2cyclictest1261036-21kworker/u8:4+events_unbound23:55:230
1011688992926,2cyclictest1213848-21kworker/u8:0+flush-259:022:35:220
1011688992923,5cyclictest1192317-21ntpq22:00:220
1011698992824,3cyclictest1188056-21kworker/u8:2+events_unbound22:10:003
1011696992822,4cyclictest1083340-21ntpq20:15:242
1011696992821,5cyclictest335557-21systemd-journal21:19:592
1011692992824,3cyclictest1302979-21apt-get23:50:131
1011692992824,3cyclictest1302979-21apt-get23:50:131
1011688992825,2cyclictest1261036-21kworker/u8:4+events_unbound00:35:210
1011688992825,2cyclictest1162402-21kworker/u8:3+events_unbound22:30:230
1011698992724,2cyclictest1048104-21kworker/u8:3+flush-259:019:45:153
1011698992724,2cyclictest1047781-21kworker/u8:0+events_unbound20:05:303
1011698992723,3cyclictest1261036-21kworker/u8:4+events_unbound23:31:363
1011698992723,3cyclictest1001618-21kworker/u8:1+events_unbound19:25:253
1011696992720,5cyclictest1109637-21ntpq20:40:242
1011692992721,4cyclictest1349573-21/usr/sbin/munin00:35:151
1011692992721,4cyclictest1338960-21latency_hist00:25:011
1011688992724,2cyclictest968583-21kworker/u8:2+events_unbound19:11:010
1011688992724,2cyclictest1038048-21kworker/u8:5+events_unbound19:35:250
1011688992723,3cyclictest1038048-21kworker/u8:5+flush-259:020:30:570
1011692992620,4cyclictest1323690-21/usr/sbin/munin00:10:211
1011688992623,2cyclictest1188056-21kworker/u8:2+events_unbound22:15:230
1011688992623,2cyclictest1162402-21kworker/u8:3+events_unbound22:56:000
1011688992623,2cyclictest1105304-21kworker/u8:0+events_unbound20:55:320
1011698992522,2cyclictest1063425-21kworker/u8:1+events_unbound20:44:013
1011698992521,3cyclictest968583-21kworker/u8:2+events_unbound19:34:023
1011698992521,3cyclictest1229585-21kworker/u8:2+events_unbound23:19:013
1011692992518,5cyclictest335557-21systemd-journal20:20:011
1011688992523,2cyclictest1063425-21kworker/u8:1+events_unbound20:25:160
1011688992522,2cyclictest1318291-21kworker/u8:1+events_unbound00:12:030
1011688992522,2cyclictest1141476-21kworker/u8:1+flush-259:021:30:160
1011688992522,2cyclictest1141476-21kworker/u8:1+flush-259:021:30:150
1011688992519,4cyclictest1213860-21latency_hist22:25:010
1011698992421,2cyclictest1131334-21kworker/u8:2+events_unbound21:20:233
1011698992421,2cyclictest1048104-21kworker/u8:3+events_unbound19:51:013
1011698992419,3cyclictest1229965-21cat22:40:023
1011696992419,3cyclictest1141476-21kworker/u8:1+events_unbound21:40:192
1011696992419,3cyclictest1105304-21kworker/u8:0+flush-259:021:05:252
1011688992421,2cyclictest1260825-21kworker/u8:0+events_unbound23:13:010
1011688992420,2cyclictest1141471-21sshd21:14:590
1011688992418,4cyclictest1026265-21ntpq19:20:200
1011698992319,3cyclictest1188056-21kworker/u8:2+events_unbound22:10:173
1011696992318,4cyclictest1229585-21kworker/u8:2+flush-259:023:30:192
1011696992317,4cyclictest1047781-21kworker/u8:0+flush-259:020:05:192
1011692992317,4cyclictest1313073-21latency_hist00:00:001
1011688992320,2cyclictest1001618-21kworker/u8:1+events_unbound19:29:020
1011698992220,2cyclictest1047781-21kworker/u8:0+events_unbound20:20:213
1011698992218,3cyclictest968583-21kworker/u8:2+flush-259:019:25:003
1011698992218,3cyclictest968583-21kworker/u8:2+events_unbound19:15:293
1011698992218,3cyclictest1172312-21kworker/u8:0+events_unbound21:54:013
1011698992218,3cyclictest1162402-21kworker/u8:3+events_unbound21:35:293
1011698992218,3cyclictest1162402-21kworker/u8:3+events_unbound21:35:283
1011698992218,3cyclictest1038048-21kworker/u8:5+events_unbound19:35:173
1011696992218,2cyclictest0-21swapper/200:29:592
1011696992217,4cyclictest1162402-21kworker/u8:3+flush-259:023:00:242
1011696992217,3cyclictest1261036-21kworker/u8:4+events_unbound23:50:192
1011696992217,3cyclictest1261036-21kworker/u8:4+events_unbound23:50:192
1011692992217,4cyclictest1141476-21kworker/u8:1+flush-259:021:35:001
1011692992217,4cyclictest1141476-21kworker/u8:1+flush-259:021:34:591
1011688992219,2cyclictest1063426-21kworker/u8:2+events_unbound20:15:540
1011688992218,3cyclictest1048104-21kworker/u8:3+events_unbound19:48:020
1011698992118,2cyclictest1339230-21apt-get00:25:123
1011698992118,2cyclictest1213848-21kworker/u8:0+flush-259:022:35:183
1011698992117,3cyclictest1047781-21kworker/u8:0+events_unbound19:57:013
1011698992117,3cyclictest1038048-21kworker/u8:5+events_unbound21:10:223
1011696992114,5cyclictest1249574-21ntpq22:55:212
1011692992117,3cyclictest1229585-21kworker/u8:2+flush-259:023:35:001
1011692992117,2cyclictest0-21swapper/120:45:011
1011692992115,4cyclictest1162402-21kworker/u8:3+flush-259:022:40:151
1011688992119,2cyclictest1261036-21kworker/u8:4+events_unbound23:50:230
1011688992119,2cyclictest1261036-21kworker/u8:4+events_unbound23:50:220
1011688992118,2cyclictest1038048-21kworker/u8:5+events_unbound20:20:540
1011688992114,5cyclictest1219036-21latency_hist22:30:000
1011698992017,2cyclictest1318291-21kworker/u8:1+events_unbound00:34:053
1011698992017,2cyclictest1229585-21kworker/u8:2+events_unbound22:45:173
1011698992015,4cyclictest335557-21systemd-journal22:00:013
1011692992014,4cyclictest1208726-21latency_hist22:20:001
1011688992015,4cyclictest1193460-21apt-get22:05:130
1011698991916,2cyclictest1016775-21idleruntime-cro19:15:013
1011698991915,3cyclictest1038048-21kworker/u8:5+flush-259:020:30:163
1011696991913,4cyclictest1100346-21/usr/sbin/munin20:35:232
1011696991912,5cyclictest1015917-21ntpq19:10:212
101169699191,16cyclictest0-21swapper/222:05:002
1011692991916,2cyclictest1136332-21idleruntime-cro21:09:591
1011692991915,3cyclictest968583-21kworker/u8:2+flush-259:019:30:191
1011692991914,3cyclictest1068843-21apt-get20:05:121
1011688991916,2cyclictest1297528-21kworker/u8:0+events_unbound00:08:020
1011698991815,2cyclictest1297528-21kworker/u8:0+events_unbound00:05:223
1011698991815,2cyclictest1261036-21kworker/u8:4+events_unbound00:00:143
1011698991814,3cyclictest1229585-21kworker/u8:2+events_unbound23:21:333
1011696991814,3cyclictest1297528-21kworker/u8:0+flush-259:023:45:172
1011696991814,3cyclictest1297528-21kworker/u8:0+flush-259:023:45:172
1011696991813,3cyclictest1203507-21idleruntime-cro22:15:012
1011696991812,4cyclictest1011580-21latency_hist19:10:002
1011696991811,5cyclictest1141469-21latency_hist21:15:012
1011692991815,2cyclictest1236249-21kthreadcore22:45:171
1011692991813,4cyclictest1100371-21apt-get20:35:121
1011688991815,2cyclictest1261036-21kworker/u8:4+events_unbound23:31:340
1011688991815,2cyclictest1255454-21kworker/u8:1+events_unbound23:09:590
1011688991815,2cyclictest1213848-21kworker/u8:0+flush-259:022:41:070
1011688991815,2cyclictest1162402-21kworker/u8:3+events_unbound23:25:150
1011698991714,2cyclictest1261036-21kworker/u8:4+events_unbound00:35:203
1011698991714,2cyclictest1177777-21kworker/u8:1+flush-259:022:00:163
1011698991713,3cyclictest1167626-21apt-get21:40:113
1011696991713,2cyclictest1276418-21idleruntime-cro23:25:002
1011696991712,3cyclictest1162402-21kworker/u8:3+flush-259:021:55:242
1011692991714,2cyclictest968583-21kworker/u8:2+flush-259:019:20:161
1011692991713,3cyclictest1141476-21kworker/u8:1+flush-259:021:40:211
1011692991713,2cyclictest0-21swapper/121:04:591
1011688991715,2cyclictest1229585-21kworker/u8:2+events_unbound23:40:200
1011688991714,2cyclictest1162402-21kworker/u8:3+events_unbound00:16:450
1011688991713,3cyclictest1255617-21idleruntime-cro23:05:000
1011688991713,3cyclictest1105304-21kworker/u8:0+events_unbound20:50:060
1011688991713,3cyclictest1001618-21kworker/u8:1+events_unbound19:41:020
1011688991711,4cyclictest1202621-21ntpq22:10:220
1011698991613,2cyclictest1229585-21kworker/u8:2+events_unbound23:06:283
1011698991613,2cyclictest1063426-21kworker/u8:2+flush-259:020:15:203
1011698991613,2cyclictest1047781-21kworker/u8:0+events_unbound20:10:153
1011698991612,3cyclictest1323681-21kworker/u8:2+events_unbound00:10:223
1011698991612,3cyclictest1229585-21kworker/u8:2+events_unbound23:35:083
1011698991610,4cyclictest1277006-21cat23:25:003
101169699169,5cyclictest1322537-21ntpq00:05:222
1011696991612,3cyclictest1229585-21kworker/u8:2+flush-259:022:45:232
1011696991612,2cyclictest1260812-21sh23:10:002
1011696991611,4cyclictest1048104-21kworker/u8:3+flush-259:019:45:252
1011696991611,3cyclictest1318291-21kworker/u8:1+flush-259:000:20:002
1011692991613,2cyclictest1302898-21latency_hist23:50:011
1011692991613,2cyclictest1302898-21latency_hist23:50:001
1011688991614,2cyclictest1260825-21kworker/u8:0+events_unbound23:15:140
1011688991614,2cyclictest1001618-21kworker/u8:1+events_unbound19:19:010
1011698991512,2cyclictest1162402-21kworker/u8:3+events_unbound22:55:213
1011698991511,3cyclictest507-21in:imuxsock21:50:013
1011696991513,1cyclictest1267268-21kthreadcore23:15:172
1011696991511,2cyclictest0-21swapper/219:40:172
1011692991512,2cyclictest0-21swapper/121:24:591
1011692991511,3cyclictest1318291-21kworker/u8:1+flush-259:000:15:181
1011688991513,1cyclictest60350irq/132-enp0s29f1:rx-021:39:150
1011688991513,1cyclictest60350irq/132-enp0s29f1:rx-021:39:140
1011688991512,2cyclictest60350irq/132-enp0s29f1:rx-021:55:220
1011698991411,2cyclictest1115126-21kworker/u8:3+events_unbound20:50:243
101169699149,4cyclictest1229387-21https22:40:012
101169299141,4cyclictest1203519-21basename22:15:011
1011692991411,2cyclictest1261492-21grep23:10:151
1011692991411,1cyclictest0-21swapper/120:55:211
1011692991410,3cyclictest1213848-21kworker/u8:0+flush-259:022:35:161
1011692991410,3cyclictest1115126-21kworker/u8:3+flush-259:020:50:361
1011688991412,1cyclictest60350irq/132-enp0s29f1:rx-021:05:340
101169899139,3cyclictest1141476-21kworker/u8:1+events_unbound21:30:143
101169899139,3cyclictest1141476-21kworker/u8:1+events_unbound21:30:133
1011698991310,2cyclictest1261036-21kworker/u8:4+events_unbound23:50:253
1011698991310,2cyclictest1261036-21kworker/u8:4+events_unbound23:50:243
1011698991310,2cyclictest1213848-21kworker/u8:0+events_unbound22:50:263
101169699139,3cyclictest1276973-21gpgv23:25:002
101169699138,3cyclictest1208966-21/usr/sbin/munin22:20:132
101169699136,4cyclictest1027120-21awk19:25:002
101169699131,3cyclictest1094560-21awk20:30:002
1011696991311,1cyclictest0-21swapper/222:10:002
1011696991311,1cyclictest0-21swapper/219:55:012
1011696991310,2cyclictest1291490-21ntp_kernel_pll_23:35:212
1011696991310,2cyclictest1121143-21cpuspeed_turbos20:55:142
101169299139,3cyclictest1162402-21kworker/u8:3+flush-259:023:20:001
101169299138,4cyclictest1177777-21kworker/u8:1+flush-259:022:05:001
101169299138,3cyclictest1250689-21apt-get23:00:111
101169299136,6cyclictest0-21swapper/122:10:011
1011692991310,2cyclictest0-21swapper/121:55:161
101168899139,1cyclictest191rcu_preempt20:10:200
1011688991310,2cyclictest1172312-21kworker/u8:0+events_unbound21:45:100
101169899129,2cyclictest1188056-21kworker/u8:2+events_unbound22:20:173
101169899128,3cyclictest1261036-21kworker/u8:4+events_unbound23:48:523
101169899128,3cyclictest1261036-21kworker/u8:4+events_unbound23:48:523
101169899128,3cyclictest1063425-21kworker/u8:1+events_unbound20:25:123
101169899126,4cyclictest335557-21systemd-journal21:09:593
101169699129,2cyclictest1313052-21cron23:59:592
101169699129,2cyclictest1058555-21cpuspeed_turbos19:55:132
101169699128,3cyclictest1141476-21kworker/u8:1+flush-259:021:30:162
101169699128,3cyclictest1141476-21kworker/u8:1+flush-259:021:30:162
101169699120,7cyclictest1318286-21idleruntime-cro00:05:002
101169299129,2cyclictest1287565-21cpuspeed_turbos23:35:121
101169299129,2cyclictest1042232-21unixbench_multi19:35:261
101169299129,2cyclictest1038048-21kworker/u8:5+flush-259:020:25:351
101169299129,2cyclictest1027795-21tr19:25:171
101169299121,10cyclictest1224090-21sshd22:34:001
101169899117,3cyclictest1318291-21kworker/u8:1+events_unbound00:20:133
101169699118,2cyclictest1240450-21cpuspeed_turbos22:50:142
101169699118,2cyclictest0-21swapper/220:10:142
101169699117,3cyclictest968583-21kworker/u8:2+flush-259:019:30:272
101169699115,4cyclictest1261356-21fw_forwarded_lo23:10:132
101169699115,4cyclictest1172825-21apt-get21:45:132
101169299118,2cyclictest1141016-21unixbench_singl21:10:271
101169299118,1cyclictest0-21swapper/100:30:201
101169299115,4cyclictest1110703-21/usr/sbin/munin20:45:161
101169299114,5cyclictest513-21gdbus23:45:001
101168899111,8cyclictest1011580-21latency_hist19:10:000
101169899107,2cyclictest1047781-21kworker/u8:0+events_unbound20:03:003
101169699108,1cyclictest0-21swapper/222:30:142
101169699107,1cyclictest0-21swapper/200:20:152
101169299107,2cyclictest1245734-21fschecks_count22:55:141
101169299106,2cyclictest1219022-21cron22:30:001
10116969996,1cyclictest0-21swapper/221:25:202
10116929995,3cyclictest1343158-21tr00:25:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional