You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 10:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sun Mar 15, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1022833994437,5cyclictest1153055-21latency_hist21:15:000
1022833994336,4cyclictest1178942-21cp21:40:010
1022846994037,2cyclictest1053727-21kworker/u8:0+flush-259:020:45:192
1022846993633,2cyclictest1256927-21kworker/u8:2+events_unbound23:05:182
1022833993530,3cyclictest1220753-21apt-get22:20:090
1022846993430,3cyclictest1137373-21kworker/u8:1+events_unbound21:20:192
1022848993327,4cyclictest1080045-21latency_hist20:04:593
1022841993328,3cyclictest1110956-21latency_hist20:35:011
1022846993226,5cyclictest1345577-21latency_hist00:20:022
1022833993128,2cyclictest1330065-21kworker/u8:0+flush-259:000:30:230
1022833993125,4cyclictest1189780-21/usr/sbin/munin21:50:140
1022833993125,4cyclictest1085168-21latency_hist20:10:010
1022841993026,3cyclictest1340656-21apt-get00:15:121
1022841993023,5cyclictest1099817-21ntpq20:20:201
1022846992925,3cyclictest1127066-21kworker/u8:4+flush-259:020:50:192
1022833992925,3cyclictest990599-21kworker/u8:2+flush-259:019:20:000
1022833992924,3cyclictest1168606-21kworker/u8:0+events_unbound21:40:120
1022848992721,4cyclictest1366238-21latency_hist00:40:003
1022846992724,2cyclictest1105829-21kworker/u8:2+flush-259:021:55:212
1022846992724,2cyclictest1043381-21kworker/u8:1+events_unbound20:20:122
1022848992623,2cyclictest1230733-21kworker/u8:0+flush-259:022:45:003
1022848992622,3cyclictest1064115-21apt-get19:50:033
1022846992622,3cyclictest1351001-21apt-get00:25:122
1022846992622,3cyclictest1189558-21kworker/u8:1+events_unbound21:50:012
1022846992620,2cyclictest1027989-21kworker/u8:0+flush-259:019:25:012
1022833992622,3cyclictest1230733-21kworker/u8:0+events_unbound22:40:190
1022833992620,4cyclictest990599-21kworker/u8:2+events_unbound19:25:170
1022846992522,2cyclictest1325163-21kworker/u8:6+events_unbound00:00:142
1022846992522,2cyclictest1314763-21kworker/u8:4+events_unbound00:10:172
1022846992522,2cyclictest1205020-21kworker/u8:3+flush-259:022:55:152
1022846992521,3cyclictest1105829-21kworker/u8:2+flush-259:021:35:212
1022833992522,2cyclictest1298624-21kworker/u8:1+flush-259:000:15:230
1022833992522,2cyclictest1105829-21kworker/u8:2+flush-259:021:30:190
1022833992521,3cyclictest1012210-21kworker/u8:3+events_unbound19:20:180
1022848992420,3cyclictest1111181-21kworker/u8:3+flush-259:020:47:263
1022848992419,4cyclictest1266361-21ntpq23:00:213
1022848992416,7cyclictest0-21swapper/319:50:003
1022846992421,2cyclictest1230733-21kworker/u8:0+events_unbound23:35:262
1022846992421,2cyclictest1012210-21kworker/u8:3+events_unbound19:55:262
1022846992420,3cyclictest1210396-21kworker/u8:4+events_unbound22:20:112
1022846992420,3cyclictest1137373-21kworker/u8:1+events_unbound21:10:182
1022833992421,2cyclictest1205020-21kworker/u8:3+flush-259:022:25:250
1022833992418,4cyclictest1106054-21apt-get20:30:120
1022848992317,4cyclictest252-21systemd-journal21:00:003
1022848992317,4cyclictest1325163-21kworker/u8:6+flush-259:000:05:173
1022848992317,4cyclictest1179687-21diskstats21:40:133
1022846992320,2cyclictest1205020-21kworker/u8:3+flush-259:022:25:152
1022846992320,2cyclictest1194934-21kworker/u8:0+flush-259:022:00:232
1022846992319,3cyclictest1325163-21kworker/u8:6+events_unbound00:20:122
1022841992319,3cyclictest252-21systemd-journal23:47:271
1022841992317,4cyclictest1168565-21latency_hist21:30:011
1022841992317,4cyclictest1147573-21cp21:10:001
1022833992320,2cyclictest1298624-21kworker/u8:1+flush-259:000:00:210
1022833992320,2cyclictest1137373-21kworker/u8:1+events_unbound21:25:010
1022848992215,5cyclictest252-21systemd-journal19:25:003
1022846992219,2cyclictest1230733-21kworker/u8:0+flush-259:023:45:232
1022846992219,2cyclictest1127066-21kworker/u8:4+flush-259:020:55:222
1022846992219,2cyclictest1043381-21kworker/u8:1+events_unbound20:00:172
1022841992218,2cyclictest1-21systemd23:25:001
1022833992219,2cyclictest1314763-21kworker/u8:4+events_unbound23:55:140
1022833992218,3cyclictest1189544-21latency_hist21:50:010
1022848992118,2cyclictest1137373-21kworker/u8:1+flush-259:021:05:163
1022848992117,3cyclictest1319980-21apt-get23:55:113
1022848992116,4cyclictest1287384-21ntpq23:20:193
1022846992118,3cyclictest1230733-21kworker/u8:0+events_unbound23:25:242
1022846992117,3cyclictest1053727-21kworker/u8:0+events_unbound20:20:002
1022841992118,2cyclictest507-21in:imuxsock19:20:001
1022841992118,2cyclictest1158236-21cron21:20:001
1022841992115,4cyclictest252-21systemd-journal22:15:181
1022833992117,3cyclictest1012210-21kworker/u8:3+flush-259:019:40:230
102283399210,19cyclictest509-21rs:main0
1022848992017,2cyclictest252-21systemd-journal00:05:003
1022848992015,4cyclictest1012210-21kworker/u8:3+flush-259:019:30:173
1022848992015,3cyclictest252-21systemd-journal19:11:183
1022846992017,2cyclictest1230733-21kworker/u8:0+flush-259:023:40:262
1022846992017,2cyclictest1105829-21kworker/u8:2+flush-259:021:05:182
1022846992017,2cyclictest1027989-21kworker/u8:0+flush-259:019:30:012
1022846992016,3cyclictest1137373-21kworker/u8:1+events_unbound21:27:202
1022841992017,2cyclictest1205020-21kworker/u8:3+flush-259:023:00:211
1022841992017,2cyclictest1205020-21kworker/u8:3+flush-259:022:40:141
1022841992016,3cyclictest1235907-21apt-get22:35:001
1022833992018,1cyclictest1230733-21kworker/u8:0+events_unbound23:25:260
1022833992017,2cyclictest1241473-21idleruntime-cro22:40:000
1022833992017,2cyclictest1053727-21kworker/u8:0+events_unbound20:25:140
1022833992016,3cyclictest1189558-21kworker/u8:1+flush-259:022:05:210
1022846991915,3cyclictest990599-21kworker/u8:2+events_unbound19:15:232
1022846991914,4cyclictest252-21systemd-journal23:52:182
1022841991916,2cyclictest1337793-21kthreadcore00:10:171
1022841991914,4cyclictest1330309-21apt-get00:05:121
1022841991912,5cyclictest252-21systemd-journal23:00:001
1022833991916,2cyclictest1325163-21kworker/u8:6+events_unbound00:25:190
1022833991916,2cyclictest1230733-21kworker/u8:0+flush-259:023:15:000
1022833991916,2cyclictest1105829-21kworker/u8:2+flush-259:021:05:210
1022833991916,2cyclictest1053727-21kworker/u8:0+flush-259:020:45:160
1022833991915,3cyclictest1230733-21kworker/u8:0+flush-259:023:05:000
1022833991915,3cyclictest1194934-21kworker/u8:0+events_unbound21:57:080
102284899188,9cyclictest1099776-21ntp_kernel_pll_20:20:193
1022846991814,3cyclictest1340622-21kworker/u8:2+flush-259:000:39:592
1022846991814,2cyclictest1023488-21irqcore19:10:172
1022846991812,4cyclictest1116713-21cat20:40:012
1022846991812,4cyclictest1116713-21cat20:40:012
1022846991811,5cyclictest1064753-21cp19:50:022
1022841991815,2cyclictest0-21swapper/122:10:011
1022841991814,3cyclictest509-21rs:main1
1022841991814,3cyclictest1053727-21kworker/u8:0+flush-259:020:25:181
1022841991811,5cyclictest1229848-21ntpq22:25:211
1022833991815,2cyclictest1205020-21kworker/u8:3+events_unbound22:45:130
1022833991815,2cyclictest1127066-21kworker/u8:4+flush-259:020:55:200
1022833991815,2cyclictest1043381-21kworker/u8:1+events_unbound20:15:220
1022833991815,1cyclictest1-21systemd23:10:000
1022833991814,3cyclictest1230733-21kworker/u8:0+events_unbound22:30:140
1022833991814,3cyclictest1205020-21kworker/u8:3+events_unbound22:15:190
102284899173,3cyclictest1361162-21latency_hist00:35:013
102284899173,3cyclictest1070485-21sed19:55:163
102284899172,3cyclictest252-21systemd-journal20:30:193
1022848991714,2cyclictest1205020-21kworker/u8:3+events_unbound22:45:173
1022848991713,3cyclictest252-21systemd-journal19:40:003
1022848991712,4cyclictest1298624-21kworker/u8:1+flush-259:000:15:173
1022848991710,4cyclictest1105790-21latency_hist20:30:003
1022846991714,2cyclictest1105829-21kworker/u8:2+events_unbound21:52:082
1022846991713,3cyclictest1043381-21kworker/u8:1+events_unbound19:45:122
1022846991712,4cyclictest1183488-21ntpq21:40:192
1022841991714,2cyclictest1202418-21kthreadcore22:00:171
1022841991713,3cyclictest1168799-21/usr/sbin/munin21:30:231
1022833991714,2cyclictest1325163-21kworker/u8:6+flush-259:000:07:230
1022833991714,2cyclictest1230733-21kworker/u8:0+events_unbound23:45:140
1022833991714,2cyclictest1111181-21kworker/u8:3+events_unbound20:40:200
1022833991714,2cyclictest1111181-21kworker/u8:3+events_unbound20:40:200
1022833991713,3cyclictest1314763-21kworker/u8:4+flush-259:023:50:190
1022833991713,3cyclictest1205020-21kworker/u8:3+flush-259:023:30:270
1022833991713,3cyclictest1137373-21kworker/u8:1+flush-259:021:15:200
1022833991713,3cyclictest1085428-21apt-get20:10:110
102284899166,1cyclictest0-21swapper/321:55:183
102284899165,1cyclictest0-21swapper/321:00:173
102284899161,3cyclictest1314115-21turbostat23:50:003
1022848991612,3cyclictest507-21in:imuxsock22:20:003
102284899160,4cyclictest1173662-21cron21:35:003
102284899160,3cyclictest1158228-21cron21:20:003
1022846991613,2cyclictest1230733-21kworker/u8:0+events_unbound22:40:232
1022846991613,2cyclictest1205020-21kworker/u8:3+flush-259:023:20:222
1022846991612,3cyclictest252-21systemd-journal00:07:182
1022846991612,3cyclictest1189558-21kworker/u8:1+events_unbound22:05:162
102284199169,5cyclictest1179370-21cat21:40:001
1022841991613,2cyclictest1074888-21idleruntime-cro19:59:591
1022841991613,1cyclictest0-21swapper/120:40:011
1022841991613,1cyclictest0-21swapper/120:40:011
1022841991612,2cyclictest0-21swapper/120:50:221
1022841991611,4cyclictest1288536-21/usr/sbin/munin23:25:141
1022833991614,1cyclictest60350irq/132-enp0s29f1:rx-022:14:450
1022833991613,2cyclictest990599-21kworker/u8:2+flush-259:019:10:220
1022833991613,2cyclictest1053727-21kworker/u8:0+flush-259:019:55:100
1022833991613,2cyclictest1043381-21kworker/u8:1+flush-259:020:00:200
1022833991612,3cyclictest1257165-21apt-get22:55:090
1022833991612,3cyclictest1230733-21kworker/u8:0+flush-259:023:38:300
102284899154,1cyclictest0-21swapper/323:05:163
102284899153,3cyclictest1319062-21cat23:50:243
102284899153,3cyclictest1238878-21kthreadcore22:35:183
102284899153,1cyclictest0-21swapper/300:25:173
102284899152,2cyclictest1335493-21apt-get00:10:123
102284899152,2cyclictest1210429-21apt-get22:10:113
102284899151,3cyclictest1256893-21idleruntime-cro22:55:003
102284899151,3cyclictest1032691-21grep19:15:263
1022848991511,3cyclictest1325163-21kworker/u8:6+flush-259:000:20:173
1022848991511,3cyclictest1137373-21kworker/u8:1+flush-259:021:20:143
102284899150,5cyclictest1236369-21cat22:35:003
102284899150,4cyclictest1184798-21cut21:45:113
102284899150,4cyclictest1116443-21touch20:40:013
102284899150,4cyclictest1054374-21grep19:40:163
102284899150,3cyclictest1116670-21gpgv20:40:013
102284899150,3cyclictest1116670-21gpgv20:40:003
1022846991513,2cyclictest1205020-21kworker/u8:3+flush-259:022:15:212
1022846991513,2cyclictest1205020-21kworker/u8:3+events_unbound23:30:022
1022846991512,2cyclictest990599-21kworker/u8:2+events_unbound19:35:152
1022846991512,2cyclictest1330065-21kworker/u8:0+events_unbound00:30:142
1022846991512,2cyclictest1205020-21kworker/u8:3+flush-259:022:45:182
1022846991511,3cyclictest1314141-21kworker/u8:2+events_unbound23:55:222
1022846991511,3cyclictest1205020-21kworker/u8:3+flush-259:023:00:202
1022846991511,3cyclictest1194934-21kworker/u8:0+events_unbound22:11:192
1022846991511,3cyclictest1043381-21kworker/u8:1+events_unbound20:27:182
102284199158,5cyclictest1141428-21ntpq21:00:221
102284199157,6cyclictest1053727-21kworker/u8:0+flush-259:021:15:001
102284199157,5cyclictest252-21systemd-journal00:40:001
1022841991511,3cyclictest1210396-21kworker/u8:4+flush-259:022:20:201
1022841991511,3cyclictest1124233-21kthreadcore20:45:161
1022833991513,1cyclictest60350irq/132-enp0s29f1:rx-021:04:380
1022833991513,1cyclictest60350irq/132-enp0s29f1:rx-000:20:240
102284899144,1cyclictest0-21swapper/322:05:163
102284899144,1cyclictest0-21swapper/321:10:173
102284899142,2cyclictest1302830-21ntpq23:35:213
102284899141,3cyclictest1293448-21cron23:30:003
102284899141,3cyclictest1276970-21grep23:10:213
102284899141,3cyclictest1192103-21kthreadcore21:50:173
1022848991411,2cyclictest1258185-21kthreadcore22:55:153
102284899140,4cyclictest1308093-21ld23:40:203
102284899140,4cyclictest1225649-21latency_hist22:25:003
102284899140,4cyclictest1089441-21grep20:10:203
102284899140,3cyclictest1282326-21smartctl23:15:263
102284899140,3cyclictest1179139-21awk21:40:013
102284899140,3cyclictest1168562-21sh21:30:003
102284899140,3cyclictest1127183-21cpuspeed_turbos20:50:123
102284899140,3cyclictest1085147-21cron20:10:003
102284699148,5cyclictest252-21systemd-journal20:35:182
102284699147,5cyclictest252-21systemd-journal21:35:002
1022846991411,2cyclictest1043381-21kworker/u8:1+flush-259:020:30:202
1022846991410,3cyclictest1053966-21apt-get19:40:112
102284199148,4cyclictest1324059-21ntpq23:55:211
102284199144,6cyclictest1038233-21basename19:25:011
1022841991411,2cyclictest858-21sshd20:10:171
1022833991413,1cyclictest60350irq/132-enp0s29f1:rx-022:02:480
1022833991412,2cyclictest60350irq/132-enp0s29f1:rx-020:54:540
1022833991412,2cyclictest1043381-21kworker/u8:1+events_unbound19:47:180
1022833991412,1cyclictest60350irq/132-enp0s29f1:rx-023:20:230
1022833991412,1cyclictest60350irq/132-enp0s29f1:rx-019:50:230
1022833991412,1cyclictest60350irq/132-enp0s29f1:rx-000:35:220
1022833991411,2cyclictest60350irq/132-enp0s29f1:rx-023:18:540
1022833991410,3cyclictest990599-21kworker/u8:2+flush-259:019:33:230
102284899131,2cyclictest0-21swapper/320:15:173
1022848991310,2cyclictest1226031-21cpuspeed_turbos22:25:133
102284899130,4cyclictest1042419-21netstat19:25:203
102284699139,3cyclictest990599-21kworker/u8:2+flush-259:019:30:292
102284699139,3cyclictest1230733-21kworker/u8:0+flush-259:022:50:262
102284699139,3cyclictest1230733-21kworker/u8:0+events_unbound22:38:202
1022846991311,1cyclictest1256927-21kworker/u8:2+events_unbound23:10:172
1022846991310,2cyclictest1137373-21kworker/u8:1+flush-259:021:15:242
102284199139,2cyclictest0-21swapper/121:20:221
102284199137,4cyclictest1236592-21apt-get22:35:121
102284199136,5cyclictest1251796-21latency_hist22:50:001
1022841991310,2cyclictest1090764-21fschecks_count20:15:151
1022841991310,2cyclictest1049064-21fschecks_time19:35:141
1022841991310,1cyclictest0-21swapper/119:35:001
102284199130,11cyclictest1027951-21awk19:15:011
102283399139,3cyclictest1205020-21kworker/u8:3+events_unbound22:53:190
102283399137,4cyclictest252-21systemd-journal23:44:180
1022833991310,2cyclictest60350irq/132-enp0s29f1:rx-000:10:180
1022833991310,2cyclictest1105829-21kworker/u8:2+flush-259:021:25:190
1022833991310,2cyclictest1053727-21kworker/u8:0+flush-259:020:35:150
102284699129,2cyclictest1205020-21kworker/u8:3+events_unbound22:34:182
102284199129,2cyclictest1318876-21ntp_offset23:50:211
102284199129,2cyclictest1189530-21idleruntime-cro21:50:001
102284199128,3cyclictest1080509-21fschecks_count20:05:131
102284199127,1cyclictest0-21swapper/119:45:471
102284199126,4cyclictest252-21systemd-journal00:32:281
102284199125,5cyclictest252-21systemd-journal23:35:001
102284199125,5cyclictest1278180-21/usr/sbin/munin23:15:181
102284199121,4cyclictest1194688-21cron21:55:001
102284699114,5cyclictest1085148-21cron20:10:002
102284199117,2cyclictest0-21swapper/123:35:171
102284199110,2cyclictest1080064-21awk20:05:001
102284199108,1cyclictest0-21swapper/120:55:191
102284199107,2cyclictest1054033-21cpuspeed_turbos19:40:131
102284199106,2cyclictest0-21swapper/121:55:171
102284199103,5cyclictest1252028-21/usr/sbin/munin22:50:161
10228419991,5cyclictest2109693-21snmpd00:20:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional