You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-24 - 04:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Wed Dec 24, 2025 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1033200994238,3cyclictest1177985-21kworker/u8:0+events_unbound21:30:252
1033200993935,3cyclictest1323024-21kworker/u8:1+events_unbound00:00:232
1033200993734,2cyclictest1229737-21kworker/u8:3+events_unbound22:25:232
1033200993731,4cyclictest252-21systemd-journal00:15:012
1033200993633,2cyclictest1022345-21kworker/u8:4+flush-259:020:40:172
1033200993633,2cyclictest1022345-21kworker/u8:4+flush-259:020:40:162
1033200993632,3cyclictest1156992-21kworker/u8:2+flush-259:021:20:202
1033198993529,4cyclictest252-21systemd-journal20:50:171
1033200993431,2cyclictest1079290-21kworker/u8:2+flush-259:020:05:182
1033204993226,4cyclictest1156189-21ntpq21:05:213
1033200993229,2cyclictest1172663-21kworker/u8:1+events_unbound22:05:232
1033194993229,2cyclictest1120535-21kworker/u8:1+flush-259:021:05:230
1033194993227,3cyclictest1038010-21kworker/u8:1+flush-259:019:30:180
1033204993125,4cyclictest1256116-21latency_hist22:45:013
1033200993128,2cyclictest1276784-21kworker/u8:5+flush-259:000:00:012
1033200993028,2cyclictest1022345-21kworker/u8:4+flush-259:020:45:192
1033200993027,2cyclictest1343567-21kworker/u8:3+events_unbound00:20:212
1033200993027,2cyclictest1291911-21kworker/u8:0+flush-259:023:30:222
1033200993027,2cyclictest1261513-21kworker/u8:2+events_unbound23:20:232
1033200993027,2cyclictest1156992-21kworker/u8:2+events_unbound21:50:212
1033200993026,3cyclictest1064144-21latency_hist19:40:012
1033194993024,4cyclictest252-21systemd-journal21:20:000
1033200992926,2cyclictest1291911-21kworker/u8:0+events_unbound23:40:222
1033200992926,2cyclictest1229737-21kworker/u8:3+flush-259:022:45:232
1033200992926,2cyclictest1022345-21kworker/u8:4+flush-259:021:25:182
1033194992925,3cyclictest1209049-21kworker/u8:3+flush-259:022:00:220
1033204992821,5cyclictest1359998-21kworker/u8:2+flush-259:000:30:183
1033200992825,2cyclictest1302255-21kworker/u8:1+flush-259:023:40:002
1033200992825,2cyclictest1099945-21kworker/u8:1+events_unbound20:25:152
1033200992825,2cyclictest1017163-21kworker/u8:3+events_unbound19:10:222
1033200992724,2cyclictest1359998-21kworker/u8:2+flush-259:000:30:192
1033200992724,2cyclictest1343567-21kworker/u8:3+flush-259:000:35:172
1033200992724,2cyclictest1022345-21kworker/u8:4+events_unbound21:15:222
1033200992724,2cyclictest1022345-21kworker/u8:4+events_unbound19:25:262
1033200992723,3cyclictest1022345-21kworker/u8:4+events_unbound19:40:212
1033198992723,3cyclictest1292559-21apt-get23:20:121
1033198992721,4cyclictest1365192-21apt-get00:30:111
1033204992622,3cyclictest1218869-21kworker/u8:2+flush-259:022:25:173
1033200992623,2cyclictest1172663-21kworker/u8:1+flush-259:021:55:162
1033194992623,2cyclictest1022345-21kworker/u8:4+flush-259:019:25:250
1033194992622,3cyclictest1323024-21kworker/u8:1+flush-259:000:00:230
1033194992620,4cyclictest1042593-21ntpq19:15:220
1033204992521,3cyclictest1156992-21kworker/u8:2+flush-259:021:40:003
1033200992522,2cyclictest1193354-21kworker/u8:0+flush-259:022:15:142
1033200992521,3cyclictest1323024-21kworker/u8:1+flush-259:000:05:212
1033200992521,3cyclictest1218869-21kworker/u8:2+flush-259:022:10:222
1033198992520,3cyclictest252-21systemd-journal20:10:171
1033198992520,3cyclictest1121137-21/usr/sbin/munin20:35:151
103319499250,23cyclictest1-21systemd19:37:170
1033204992419,3cyclictest1094757-21kworker/u8:0+flush-259:020:45:173
1033200992421,2cyclictest1245626-21kworker/u8:4+flush-259:022:40:202
1033200992421,2cyclictest1229737-21kworker/u8:3+flush-259:022:25:002
1033200992421,2cyclictest1120535-21kworker/u8:1+events_unbound21:10:252
1033200992420,3cyclictest1048637-21latency_hist19:25:012
1033200992320,2cyclictest1022345-21kworker/u8:4+flush-259:020:40:002
1033198992318,3cyclictest1178393-21apt-get21:30:131
1033198992317,4cyclictest1131196-21latency_hist20:45:011
1033204992218,3cyclictest1359998-21kworker/u8:2+flush-259:000:30:003
1033204992216,4cyclictest252-21systemd-journal23:55:013
1033204992216,4cyclictest1271628-21latency_hist23:00:013
1033204992216,4cyclictest1271628-21latency_hist23:00:013
1033204992216,4cyclictest1214542-21/usr/sbin/munin22:05:243
1033200992219,2cyclictest1276784-21kworker/u8:5+events_unbound23:05:132
1033200992219,2cyclictest1261513-21kworker/u8:2+flush-259:023:10:262
1033200992219,2cyclictest1084250-21kworker/u8:3+flush-259:020:10:202
1033198992218,3cyclictest1017163-21kworker/u8:3+flush-259:019:20:161
1033198992217,3cyclictest1287133-21latency_hist23:15:011
1033194992216,4cyclictest1126277-21/usr/sbin/munin20:40:230
1033194992216,4cyclictest1126277-21/usr/sbin/munin20:40:230
1033204992116,3cyclictest1094757-21kworker/u8:0+flush-259:020:25:013
1033204992114,5cyclictest1359774-21latency_hist00:25:013
1033200992118,2cyclictest1276784-21kworker/u8:5+events_unbound00:15:012
1033200992118,2cyclictest1188137-21kworker/u8:4+events_unbound21:40:212
1033200992118,2cyclictest1022345-21kworker/u8:4+flush-259:021:00:202
1033200992115,4cyclictest1054062-21/usr/sbin/munin19:30:142
1033200992112,6cyclictest252-21systemd-journal19:20:002
1033198992115,4cyclictest1043477-21latency_hist19:20:011
1033194992118,2cyclictest1261513-21kworker/u8:2+flush-259:023:10:160
1033194992118,2cyclictest1141555-21kworker/u8:3+flush-259:020:55:180
1033194992117,3cyclictest1276784-21kworker/u8:5+flush-259:023:55:200
1033194992117,3cyclictest1276784-21kworker/u8:5+flush-259:000:20:210
1033194992117,3cyclictest1218869-21kworker/u8:2+flush-259:022:10:280
1033204992016,3cyclictest1094757-21kworker/u8:0+flush-259:020:25:183
1033204992015,3cyclictest1349726-21/usr/sbin/munin00:15:133
1033200992017,2cyclictest1177985-21kworker/u8:0+events_unbound21:35:182
1033200992017,2cyclictest1099945-21kworker/u8:1+flush-259:020:19:592
1033200992017,2cyclictest1074076-21kworker/u8:1+events_unbound19:50:202
1033198992017,2cyclictest1218869-21kworker/u8:2+flush-259:022:10:221
1033198992017,2cyclictest1094128-21needreboot20:05:201
1033198992015,3cyclictest1370378-21/usr/sbin/munin00:35:131
1033194992016,3cyclictest1302255-21kworker/u8:1+flush-259:023:30:000
1033204991916,2cyclictest1261513-21kworker/u8:2+flush-259:023:10:183
1033204991913,4cyclictest1266439-21latency_hist22:55:013
1033200991915,3cyclictest1323024-21kworker/u8:1+events_unbound23:50:132
1033200991915,3cyclictest1079290-21kworker/u8:2+events_unbound19:55:282
1033198991914,4cyclictest1229737-21kworker/u8:3+flush-259:023:00:311
1033194991916,2cyclictest1261513-21kworker/u8:2+flush-259:023:00:270
1033194991916,2cyclictest1261513-21kworker/u8:2+events_unbound22:55:230
1033194991916,2cyclictest1261513-21kworker/u8:2+events_unbound22:55:230
1033194991916,2cyclictest1156992-21kworker/u8:2+flush-259:021:45:200
1033194991915,3cyclictest1120535-21kworker/u8:1+events_unbound20:35:240
1033194991915,3cyclictest1094757-21kworker/u8:0+events_unbound20:45:160
1033194991914,4cyclictest1141588-21latency_hist20:55:010
1033194991914,3cyclictest1156992-21kworker/u8:2+flush-259:021:15:010
1033194991913,4cyclictest1078743-21ntpq19:50:240
1033204991814,3cyclictest1244868-21ntp_kernel_err22:30:213
1033204991813,3cyclictest1297717-21apt-get23:25:113
1033204991813,3cyclictest1059223-21/usr/sbin/munin19:35:203
1033204991812,4cyclictest1312933-21latency_hist23:40:003
1033200991816,2cyclictest1291911-21kworker/u8:0+events_unbound23:45:252
1033200991815,3cyclictest1261513-21kworker/u8:2+events_unbound23:25:252
1033200991815,2cyclictest1043500-21kworker/u8:2+events_unbound19:45:262
1033200991814,3cyclictest1141555-21kworker/u8:3+events_unbound20:55:122
1033200991813,3cyclictest1110172-21kworker/u8:2+events_unbound20:30:202
103319899181,8cyclictest1292303-21awk23:20:001
1033194991814,3cyclictest1022345-21kworker/u8:4+flush-259:019:55:200
1033194991814,3cyclictest1017163-21kworker/u8:3+flush-259:019:20:250
1033204991714,2cyclictest0-21swapper/323:05:003
1033204991713,3cyclictest1229737-21kworker/u8:3+flush-259:022:45:173
1033204991713,3cyclictest1094757-21kworker/u8:0+events_unbound20:30:173
1033204991711,4cyclictest1328840-21diskstats23:55:163
1033200991714,2cyclictest1261513-21kworker/u8:2+events_unbound23:00:232
1033200991714,2cyclictest1229737-21kworker/u8:3+events_unbound22:33:122
1033200991713,3cyclictest1193898-21/usr/sbin/munin21:45:222
1033200991711,2cyclictest1359998-21kworker/u8:2+events_unbound00:25:262
1033198991713,2cyclictest0-21swapper/123:45:001
1033194991714,2cyclictest60350irq/132-enp0s29f1:rx-020:10:190
1033194991714,2cyclictest1172663-21kworker/u8:1+flush-259:022:15:260
1033194991714,2cyclictest1156992-21kworker/u8:2+flush-259:021:50:190
1033194991713,3cyclictest1209049-21kworker/u8:3+flush-259:022:05:190
1033204991613,2cyclictest1276784-21kworker/u8:5+flush-259:023:30:263
1033204991612,3cyclictest1374457-21ntp_kernel_pll_00:35:203
1033200991613,2cyclictest1245582-21kworker/u8:1+events_unbound22:50:192
1033200991613,2cyclictest1172663-21kworker/u8:1+flush-259:022:01:422
1033198991611,3cyclictest1022345-21kworker/u8:4+flush-259:020:15:311
1033194991614,1cyclictest60350irq/132-enp0s29f1:rx-021:55:170
1033194991614,1cyclictest60350irq/132-enp0s29f1:rx-020:07:330
1033194991614,1cyclictest60350irq/132-enp0s29f1:rx-000:32:050
1033194991613,2cyclictest61150irq/133-enp0s29f1:tx-023:05:200
1033194991613,2cyclictest60350irq/132-enp0s29f1:rx-020:30:220
1033194991613,2cyclictest60350irq/132-enp0s29f1:rx-019:11:520
1033194991613,2cyclictest60350irq/132-enp0s29f1:rx-000:35:230
1033194991613,2cyclictest1318073-21turbostat23:45:000
1033194991613,2cyclictest1291911-21kworker/u8:0+flush-259:023:50:250
1033194991613,2cyclictest1043500-21kworker/u8:2+events_unbound19:45:280
1033194991612,3cyclictest1172663-21kworker/u8:1+flush-259:021:30:180
1033194991612,3cyclictest1099945-21kworker/u8:1+events_unbound20:25:230
1033204991512,2cyclictest1297455-21idleruntime-cro23:25:003
1033204991512,2cyclictest1156992-21kworker/u8:2+flush-259:021:50:173
1033204991512,1cyclictest0-21swapper/321:50:013
1033204991511,3cyclictest1333845-21apt-get00:00:123
1033204991511,3cyclictest1161842-21kworker/u8:3+flush-259:021:15:013
1033200991511,3cyclictest1120535-21kworker/u8:1+events_unbound20:51:192
103319899158,5cyclictest1109718-21ntpq20:20:251
1033198991512,2cyclictest0-21swapper/121:10:011
1033198991511,3cyclictest1079290-21kworker/u8:2+flush-259:019:55:251
1033198991510,3cyclictest252-21systemd-journal23:58:061
1033198991510,3cyclictest1178126-21latency_hist21:30:011
103319499159,4cyclictest252-21systemd-journal23:36:180
1033194991514,1cyclictest60350irq/132-enp0s29f1:rx-023:20:040
1033194991514,1cyclictest60350irq/132-enp0s29f1:rx-000:11:080
1033194991513,2cyclictest1229737-21kworker/u8:3+events_unbound22:50:120
1033194991513,1cyclictest60350irq/132-enp0s29f1:rx-023:45:170
1033194991513,1cyclictest60350irq/132-enp0s29f1:rx-023:18:470
1033194991513,1cyclictest60350irq/132-enp0s29f1:rx-022:20:150
1033194991513,1cyclictest60350irq/132-enp0s29f1:rx-020:20:260
1033194991513,1cyclictest60350irq/132-enp0s29f1:rx-020:15:590
1033194991513,1cyclictest60350irq/132-enp0s29f1:rx-000:29:430
1033194991513,1cyclictest60350irq/132-enp0s29f1:rx-000:15:150
1033194991512,2cyclictest60350irq/132-enp0s29f1:rx-021:20:210
1033194991512,2cyclictest60350irq/132-enp0s29f1:rx-020:02:200
1033194991512,2cyclictest1343884-21turbostat00:10:000
1033194991511,3cyclictest1156992-21kworker/u8:2+events_unbound21:25:120
1033194991510,4cyclictest1064407-21apt-get19:40:100
103320499148,4cyclictest509-21rs:main3
1033204991412,1cyclictest0-21swapper/323:10:003
1033204991411,2cyclictest1074933-21fschecks_time19:50:163
1033204991410,2cyclictest0-21swapper/321:25:003
1033200991411,3cyclictest1245582-21kworker/u8:1+flush-259:022:35:052
1033200991410,3cyclictest1094757-21kworker/u8:0+events_unbound20:20:242
1033200991410,3cyclictest1022345-21kworker/u8:4+events_unbound21:08:172
103319899149,3cyclictest1219455-21apt-get22:10:011
103319899148,4cyclictest252-21systemd-journal21:00:001
1033198991411,2cyclictest1111231-21irqcore20:25:171
1033198991410,3cyclictest1156992-21kworker/u8:2+flush-259:021:40:251
1033198991410,2cyclictest1077016-21kthreadcore19:50:201
1033194991412,1cyclictest60350irq/132-enp0s29f1:rx-021:40:140
1033194991411,2cyclictest60350irq/132-enp0s29f1:rx-022:30:190
1033194991411,2cyclictest1240613-21turbostat22:29:590
1033194991410,3cyclictest1302255-21kworker/u8:1+flush-259:023:34:010
103320499139,3cyclictest1022345-21kworker/u8:4+events_unbound21:00:233
1033204991311,1cyclictest1227638-21kthreadcore22:15:193
1033204991311,1cyclictest1084738-21idleruntime-cro20:00:003
103319899139,3cyclictest1250769-21journal-offline22:38:011
103319899139,3cyclictest1161798-21ntp_kernel_pll_21:10:241
103319899139,3cyclictest1131460-21apt-get20:45:111
1033198991311,1cyclictest0-21swapper/121:55:011
1033198991310,2cyclictest1255202-21ntp_kernel_err22:40:201
1033198991310,2cyclictest1240873-21apt-get22:30:131
1033198991310,2cyclictest1022345-21kworker/u8:4+flush-259:021:17:211
103319499139,3cyclictest1022345-21kworker/u8:4+flush-259:021:03:190
1033194991311,2cyclictest60350irq/132-enp0s29f1:rx-022:45:050
1033194991311,2cyclictest1177985-21kworker/u8:0+events_unbound21:35:210
1033194991311,1cyclictest60350irq/132-enp0s29f1:rx-022:35:180
103320499129,2cyclictest1121782-21irqcore20:35:183
1033204991210,1cyclictest0-21swapper/319:40:163
1033204991210,1cyclictest0-21swapper/319:20:163
103320099128,3cyclictest1266769-21cpuspeed_turbos22:55:132
103320099128,3cyclictest1266769-21cpuspeed_turbos22:55:122
103319899129,2cyclictest1325809-21kthreadcore23:50:181
103319899129,2cyclictest0-21swapper/100:15:011
103319899128,2cyclictest1053833-21munin-plugin-st19:30:011
103319899126,4cyclictest1349065-21apt-get00:15:011
1033198991210,1cyclictest1364925-21sh00:29:591
103320499119,1cyclictest0-21swapper/321:25:133
103320499119,1cyclictest0-21swapper/319:45:303
103320499118,2cyclictest0-21swapper/322:10:213
103320499118,2cyclictest0-21swapper/321:55:183
103320499116,4cyclictest1074076-21kworker/u8:1+flush-259:020:00:233
103320499115,4cyclictest1182503-21ntp_states21:30:253
103319899119,1cyclictest0-21swapper/120:40:151
103319899118,1cyclictest0-21swapper/122:15:171
103319899118,1cyclictest0-21swapper/121:05:001
103319899116,3cyclictest1058992-21latency_hist19:35:001
103319899110,2cyclictest1261278-21awk22:50:001
1033194991110,1cyclictest60350irq/132-enp0s29f1:rx-022:41:160
103320499108,1cyclictest0-21swapper/320:15:203
103320499106,3cyclictest1323024-21kworker/u8:1+flush-259:000:05:193
103320499106,3cyclictest1291911-21kworker/u8:0+flush-259:023:45:193
103320499106,2cyclictest0-21swapper/320:15:013
103320499104,4cyclictest1017163-21kworker/u8:3+flush-259:019:10:173
10331989997,1cyclictest0-21swapper/100:05:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional