You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-01 - 22:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Sat Aug 01, 2026 12:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
931399995140,9cyclictest967918-21latency_hist07:45:012
931396995042,5cyclictest335557-21systemd-journal09:28:011
931402994945,3cyclictest963009-21kworker/u8:3+flush-259:007:54:593
931396994441,2cyclictest1244177-21kworker/u8:3+flush-259:012:26:041
931396994440,3cyclictest1066416-21gpgv09:20:011
931396994438,4cyclictest1092567-21latency_hist09:44:591
931402994339,3cyclictest1207536-21kworker/u8:2+flush-259:011:45:013
931396994340,2cyclictest1186917-21kworker/u8:0+flush-259:011:35:021
931399994234,5cyclictest1050733-21latency_hist09:05:002
931399994137,3cyclictest3060545-21kworker/u8:5+flush-259:007:50:002
931396994138,2cyclictest963009-21kworker/u8:3+events_unbound08:25:011
931391994134,5cyclictest335557-21systemd-journal11:05:000
931402994033,5cyclictest1045583-21latency_hist09:00:013
931399994032,6cyclictest1171187-21cat11:00:012
931396994037,2cyclictest1004451-21kworker/u8:4+events_unbound08:25:191
931402993933,4cyclictest335557-21systemd-journal09:10:293
931391993834,3cyclictest1212945-21kworker/u8:1+flush-259:012:35:000
931396993732,4cyclictest335557-21systemd-journal10:50:001
931391993733,3cyclictest1238811-21kworker/u8:2+flush-259:012:05:000
931402993631,3cyclictest1071893-21kworker/u8:0+events_unbound09:47:043
931396993634,2cyclictest1228902-21kworker/u8:4+events_unbound12:36:021
931396993630,4cyclictest335557-21systemd-journal07:36:021
931391993630,4cyclictest942377-21/usr/sbin/munin07:20:130
93139199360,35cyclictest1-21systemd09:34:030
931399993529,4cyclictest1171190-21kworker/u8:2+flush-259:011:15:012
931399993528,4cyclictest1217894-21latency_hist11:45:012
931402993430,3cyclictest1071893-21kworker/u8:0+events_unbound09:40:333
931402993427,5cyclictest1192052-21latency_hist11:20:013
931396993432,2cyclictest973144-21kworker/u8:2+flush-259:008:45:231
931402993330,2cyclictest1045343-21kworker/u8:0+events_unbound09:00:263
931402993327,4cyclictest1207507-21latency_hist11:35:013
931396993228,3cyclictest952688-21/usr/sbin/munin07:30:161
931391993226,4cyclictest1123613-21latency_hist10:15:010
931402993127,3cyclictest1025101-21kworker/u8:1+events_unbound09:05:043
931399993125,4cyclictest942239-21cat07:20:002
931399993124,5cyclictest1228838-21cat11:55:002
931396993129,2cyclictest3060545-21kworker/u8:5+events_unbound07:15:041
931402993027,2cyclictest1123628-21kworker/u8:4+events_unbound10:31:033
931399993024,4cyclictest1176997-21diskstats11:05:152
931396993027,2cyclictest963009-21kworker/u8:3+events_unbound08:02:031
931396993027,2cyclictest1207536-21kworker/u8:2+events_unbound11:54:041
931396993027,2cyclictest1045343-21kworker/u8:0+events_unbound09:00:161
931396993026,3cyclictest335557-21systemd-journal11:15:001
931391993027,2cyclictest1149874-21kworker/u8:2+events_unbound10:41:040
931391993021,6cyclictest1181720-21latency_hist11:10:010
931399992921,5cyclictest509-21rs:main2
931396992926,3cyclictest1228902-21kworker/u8:4+events_unbound12:10:531
931396992925,3cyclictest973144-21kworker/u8:2+flush-259:008:20:001
931396992925,3cyclictest3060545-21kworker/u8:5+flush-259:007:24:591
931391992923,4cyclictest1275312-21latency_hist12:40:010
931391992922,5cyclictest1066300-21latency_hist09:20:010
931402992822,4cyclictest335557-21systemd-journal09:18:013
931396992825,2cyclictest1176801-21kworker/u8:3+events_unbound11:05:191
931402992724,3cyclictest3060545-21kworker/u8:5+events_unbound07:26:023
931402992724,2cyclictest1071893-21kworker/u8:0+events_unbound09:25:323
931402992723,3cyclictest3060545-21kworker/u8:5+events_unbound07:24:043
931402992723,3cyclictest1097757-21kworker/u8:4+flush-259:009:55:193
931399992721,4cyclictest1009834-21diskstats08:25:152
931396992725,2cyclictest973144-21kworker/u8:2+flush-259:009:10:251
931396992724,2cyclictest1160219-21kworker/u8:0+events_unbound10:50:181
931396992723,3cyclictest1228286-21kworker/u8:0+flush-259:012:00:001
931391992722,3cyclictest957890-21apt-get07:35:120
931391992721,4cyclictest509-21rs:main0
931391992720,5cyclictest1102018-21ntpq09:50:200
931402992623,2cyclictest973144-21kworker/u8:2+events_unbound08:29:033
931402992623,2cyclictest1004451-21kworker/u8:4+flush-259:008:30:203
931402992621,4cyclictest1082498-21apt-get09:35:123
931399992621,4cyclictest994151-21apt-get08:10:132
931396992623,2cyclictest952468-21kworker/u8:1+flush-259:007:55:111
931396992623,2cyclictest952468-21kworker/u8:1+events_unbound08:10:181
931396992622,3cyclictest1097757-21kworker/u8:4+events_unbound09:56:041
931396992621,3cyclictest1025111-21/usr/sbin/munin08:40:151
931402992522,2cyclictest1207536-21kworker/u8:2+events_unbound11:50:533
931402992522,2cyclictest1019736-21kworker/u8:0+flush-259:008:35:163
931396992522,2cyclictest1123627-21kworker/u8:3+events_unbound10:40:251
931402992421,2cyclictest952468-21kworker/u8:1+events_unbound08:05:033
931402992420,3cyclictest973144-21kworker/u8:2+events_unbound08:22:033
931402992420,3cyclictest1244177-21kworker/u8:3+flush-259:012:25:573
931396992422,2cyclictest1108047-21kworker/u8:1+flush-259:011:00:241
931396992422,2cyclictest1025101-21kworker/u8:1+events_unbound08:51:031
931396992421,2cyclictest973144-21kworker/u8:2+flush-259:008:30:241
931396992421,2cyclictest1228286-21kworker/u8:0+flush-259:012:20:241
931396992421,2cyclictest1212945-21kworker/u8:1+events_unbound11:40:221
931396992421,2cyclictest1108047-21kworker/u8:1+flush-259:010:25:251
931391992418,4cyclictest1244397-21/usr/sbin/munin12:10:150
931402992320,3cyclictest1171190-21kworker/u8:2+events_unbound11:12:043
931402992320,2cyclictest1197111-21kworker/u8:3+events_unbound11:35:193
931402992319,3cyclictest952468-21kworker/u8:1+events_unbound07:45:183
931402992319,3cyclictest1228902-21kworker/u8:4+flush-259:012:15:163
931402992319,3cyclictest1108047-21kworker/u8:1+events_unbound10:23:033
931399992318,3cyclictest1102749-40mandb09:53:202
931399992316,5cyclictest1175701-21ntpq11:00:212
931399992314,6cyclictest1253637-21ntpq12:15:222
931396992321,1cyclictest509-21rs:main1
931396992320,2cyclictest973144-21kworker/u8:2+events_unbound10:10:251
931396992320,2cyclictest1212945-21kworker/u8:1+events_unbound12:09:031
931396992320,2cyclictest1186917-21kworker/u8:0+events_unbound11:20:181
931396992320,2cyclictest1123627-21kworker/u8:3+events_unbound10:38:011
931391992319,3cyclictest1160328-21gpgv10:50:010
931391992318,4cyclictest335557-21systemd-journal10:30:000
931391992318,3cyclictest1-21systemd11:29:020
931402992217,3cyclictest1123627-21kworker/u8:3+events_unbound10:40:193
931399992219,2cyclictest1192044-21idleruntime-cro11:20:002
931399992215,5cyclictest1274430-21ntpq12:35:212
931396992219,2cyclictest1108047-21kworker/u8:1+flush-259:010:15:211
931396992219,2cyclictest1108047-21kworker/u8:1+events_unbound10:05:191
931396992217,4cyclictest1197438-21/usr/sbin/munin11:25:201
931391992219,2cyclictest1187142-21apt-get11:15:120
931391992216,4cyclictest335557-21systemd-journal08:35:260
931391992216,4cyclictest1196292-21ntpq11:20:210
931391992216,4cyclictest1148977-21ntpq10:35:250
931402992118,2cyclictest952468-21kworker/u8:1+events_unbound07:35:243
931402992118,2cyclictest1228902-21kworker/u8:4+flush-259:012:00:243
931402992117,3cyclictest931375-21kworker/u8:2+events_unbound07:18:033
931402992116,3cyclictest1160810-21base6410:50:013
93139999219,9cyclictest0-21swapper/212:05:132
93139999218,11cyclictest0-21swapper/208:50:002
931399992117,3cyclictest973144-21kworker/u8:2+flush-259:008:55:202
931399992115,4cyclictest1025101-21kworker/u8:1+flush-259:008:50:172
931399992110,10cyclictest1207508-21idleruntime-cro11:35:002
931396992118,2cyclictest1108047-21kworker/u8:1+events_unbound10:00:041
931396992118,2cyclictest1019736-21kworker/u8:0+flush-259:008:35:211
931396992117,3cyclictest973144-21kworker/u8:2+flush-259:008:55:251
931396992117,3cyclictest1197111-21kworker/u8:3+events_unbound11:48:031
931391992117,3cyclictest1050969-21kworker/u8:3+flush-259:009:05:240
931402992017,2cyclictest1228902-21kworker/u8:4+events_unbound12:35:043
931402992017,2cyclictest1228902-21kworker/u8:4+events_unbound12:10:203
931402992016,3cyclictest1239218-21/usr/sbin/munin12:05:233
931402992016,3cyclictest1228286-21kworker/u8:0+events_unbound12:20:273
931402992013,6cyclictest500-21systemd-logind08:45:013
93139999209,9cyclictest947673-21cpuspeed_turbos07:25:122
931399992015,3cyclictest1192745-21irqcore11:20:162
931399992014,4cyclictest1155300-21apt-get10:45:122
931396992017,2cyclictest1050969-21kworker/u8:3+flush-259:009:20:281
931396992016,3cyclictest973144-21kworker/u8:2+events_unbound09:53:271
931396992016,3cyclictest3060545-21kworker/u8:5+events_unbound07:25:131
931391992015,3cyclictest335557-21systemd-journal10:06:010
931402991916,2cyclictest973144-21kworker/u8:2+events_unbound08:15:243
931402991915,3cyclictest952468-21kworker/u8:1+events_unbound08:14:043
931402991915,3cyclictest1186917-21kworker/u8:0+events_unbound11:20:023
931402991915,3cyclictest1149874-21kworker/u8:2+events_unbound10:45:183
93139999199,8cyclictest0-21swapper/210:10:232
93139999191,5cyclictest1124727-21irqcore10:15:172
931399991913,4cyclictest1113684-21diskstats10:05:142
931396991916,2cyclictest952468-21kworker/u8:1+events_unbound07:45:261
931396991916,2cyclictest1050969-21kworker/u8:3+events_unbound09:05:201
931396991915,3cyclictest1108047-21kworker/u8:1+events_unbound10:30:201
931396991913,4cyclictest1238118-21ntpq12:00:211
931396991913,4cyclictest1077323-21apt-get09:30:121
931391991914,4cyclictest947579-21apt-get07:25:120
931391991913,4cyclictest335557-21systemd-journal10:20:000
931402991815,2cyclictest1123627-21kworker/u8:3+events_unbound10:55:473
931402991814,3cyclictest973144-21kworker/u8:2+events_unbound10:10:153
931402991814,3cyclictest1212945-21kworker/u8:1+events_unbound11:55:153
931402991814,3cyclictest1108047-21kworker/u8:1+events_unbound11:25:233
93139999187,9cyclictest1072199-21cpuspeed_turbos09:25:142
93139999186,1cyclictest0-21swapper/209:35:262
931399991815,2cyclictest1066348-21apt-config09:20:002
931399991815,2cyclictest0-21swapper/210:45:002
93139999181,4cyclictest988731-21apt08:05:002
93139999180,4cyclictest1234534-21grep12:00:162
931396991814,3cyclictest920539-21kworker/u8:0+flush-259:007:10:221
931396991814,3cyclictest1228902-21kworker/u8:4+events_unbound12:32:021
931391991815,2cyclictest1030338-21apt-get08:45:120
931391991814,3cyclictest60350irq/132-enp0s29f1:rx-012:15:230
931391991812,4cyclictest335557-21systemd-journal07:20:000
931402991715,2cyclictest3060545-21kworker/u8:5+events_unbound07:40:013
931402991714,2cyclictest952468-21kworker/u8:1+flush-259:008:05:013
931402991714,2cyclictest1108047-21kworker/u8:1+events_unbound11:00:433
931402991714,2cyclictest1108047-21kworker/u8:1+events_unbound10:17:043
931402991713,3cyclictest3060545-21kworker/u8:5+flush-259:007:30:263
931402991713,3cyclictest1212945-21kworker/u8:1+events_unbound11:45:143
931402991713,3cyclictest1050969-21kworker/u8:3+events_unbound09:20:153
93139999175,1cyclictest0-21swapper/210:25:252
93139999174,3cyclictest513-21gdbus12:35:002
93139999174,3cyclictest1260500-21sed12:25:192
93139999174,2cyclictest0-21swapper/210:20:282
93139999171,5cyclictest509-21rs:main2
93139999171,4cyclictest1004225-21munin-plugin-st08:20:002
931399991712,3cyclictest507-21in:imuxsock09:10:012
93139999170,4cyclictest957209-21head07:30:272
93139999170,4cyclictest1218570-21irqcore11:45:182
93139999170,4cyclictest1092566-21turbostat.cron09:45:002
931396991712,3cyclictest1-21systemd11:40:011
931391991714,2cyclictest60350irq/132-enp0s29f1:rx-011:00:000
931391991714,2cyclictest60350irq/132-enp0s29f1:rx-008:27:290
931391991713,2cyclictest1197111-21kworker/u8:3+events_unbound11:35:230
931391991712,4cyclictest973144-21kworker/u8:2+events_unbound08:00:140
931402991612,3cyclictest1171190-21kworker/u8:2+events_unbound11:05:033
931402991611,4cyclictest335557-21systemd-journal10:00:033
93139999162,2cyclictest1165802-21awk10:55:002
93139999161,8cyclictest1009418-21awk08:24:592
93139999160,5cyclictest941971-21latency_hist07:20:012
93139999160,5cyclictest1228247-21munin-run11:54:592
931396991613,2cyclictest963009-21kworker/u8:3+events_unbound08:07:031
931396991613,2cyclictest952468-21kworker/u8:1+flush-259:007:50:151
931396991613,2cyclictest1171190-21kworker/u8:2+flush-259:011:15:261
931391991613,2cyclictest60350irq/132-enp0s29f1:rx-011:47:550
931391991613,2cyclictest1123627-21kworker/u8:3+flush-259:010:20:270
931391991612,3cyclictest1071012-21ntp_offset09:20:210
931402991513,2cyclictest1108047-21kworker/u8:1+events_unbound10:05:233
931402991512,2cyclictest952468-21kworker/u8:1+events_unbound07:55:123
931402991512,2cyclictest920539-21kworker/u8:0+flush-259:007:10:193
931402991510,3cyclictest1265134-21/usr/sbin/munin12:30:243
93139999156,1cyclictest0-21swapper/209:59:592
93139999155,3cyclictest401ktimers/209:20:212
93139999151,4cyclictest936579-21systemd-run07:15:002
93139999151,2cyclictest1145377-21ps10:35:202
93139199158,5cyclictest999066-21latency_hist08:15:010
931391991513,1cyclictest60350irq/132-enp0s29f1:rx-011:14:370
931391991513,1cyclictest60350irq/132-enp0s29f1:rx-009:11:530
931391991513,1cyclictest60350irq/132-enp0s29f1:rx-009:00:230
931391991513,1cyclictest60350irq/132-enp0s29f1:rx-007:53:200
931391991512,2cyclictest60350irq/132-enp0s29f1:rx-011:50:200
931391991512,2cyclictest60350irq/132-enp0s29f1:rx-011:30:130
931391991512,2cyclictest60350irq/132-enp0s29f1:rx-009:26:040
931391991512,2cyclictest1103593-21irqcore09:55:190
931391991511,3cyclictest1025359-21fschecks_time08:40:140
931391991511,2cyclictest60350irq/132-enp0s29f1:rx-008:15:200
931402991410,3cyclictest1050969-21kworker/u8:3+events_unbound09:51:043
93139999141,2cyclictest1029212-21ntpq08:40:222
931399991410,3cyclictest1197111-21kworker/u8:3+flush-259:011:35:222
93139699148,4cyclictest1087366-21latency_hist09:40:001
93139199148,4cyclictest1143861-21ntpq10:30:220
931391991413,1cyclictest60350irq/132-enp0s29f1:rx-010:02:080
931391991412,1cyclictest60350irq/132-enp0s29f1:rx-012:05:240
931391991412,1cyclictest60350irq/132-enp0s29f1:rx-010:50:160
931391991412,1cyclictest60350irq/132-enp0s29f1:rx-009:42:440
931391991412,1cyclictest60350irq/132-enp0s29f1:rx-008:50:250
931391991412,1cyclictest60350irq/132-enp0s29f1:rx-008:20:140
931391991412,1cyclictest60350irq/132-enp0s29f1:rx-008:09:340
931391991412,1cyclictest60350irq/132-enp0s29f1:rx-007:40:290
93139999136,4cyclictest973562-21diskstats07:50:152
93139999131,4cyclictest978752-21df_inode07:55:132
93139999130,4cyclictest1061086-21idleruntime-cro09:15:012
931396991310,2cyclictest1123627-21kworker/u8:3+events_unbound10:20:241
931391991311,1cyclictest60350irq/132-enp0s29f1:rx-009:35:160
931391991311,1cyclictest60350irq/132-enp0s29f1:rx-007:10:210
931402991212,0cyclictest1040453-21kworker/3:3+events_power_efficient10:35:283
93139199129,2cyclictest60350irq/132-enp0s29f1:rx-007:55:180
931391991210,2cyclictest60350irq/132-enp0s29f1:rx-011:55:180
93140299118,2cyclictest1037814-21sort08:50:173
93140299117,3cyclictest973144-21kworker/u8:2+flush-259:008:46:193
93139199119,1cyclictest60350irq/132-enp0s29f1:rx-009:45:230
931391991110,1cyclictest60350irq/132-enp0s29f1:rx-007:45:200
93140299107,2cyclictest1134727-21cpuspeed_turbos10:25:133
93140299105,3cyclictest490-21dbus-daemon09:34:593
93139699106,3cyclictest1212945-21kworker/u8:1+events_unbound12:16:551
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional