You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 04:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Tue May 26, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
928458994643,2cyclictest1073899-21kworker/u8:0+events_unbound21:35:243
928458994542,2cyclictest1219688-21kworker/u8:1+events_unbound00:05:243
928450994539,4cyclictest954218-21latency_hist19:35:001
928453994441,2cyclictest1173268-21kworker/u8:4+flush-259:023:55:202
928453994339,3cyclictest869868-21kworker/u8:1+events_unbound19:40:102
928458994138,2cyclictest1130883-21kworker/u8:3+events_unbound23:20:223
928458994138,2cyclictest1073899-21kworker/u8:0+events_unbound22:00:223
928453994138,2cyclictest1130883-21kworker/u8:3+flush-259:022:40:012
928453994138,2cyclictest1130883-21kworker/u8:3+flush-259:022:40:002
928453994138,2cyclictest1110088-21kworker/u8:2+flush-259:022:10:012
928448994137,2cyclictest211rcuc/020:45:000
928458994037,2cyclictest965617-21kworker/u8:4+events_unbound20:00:163
928458994035,4cyclictest1001666-21latency_hist20:20:003
928450994037,2cyclictest1053110-21kworker/u8:3+events_unbound21:40:161
928458993935,3cyclictest1110301-21/usr/sbin/munin22:05:233
928453993936,2cyclictest1193647-21kworker/u8:2+events_unbound23:30:152
928450993934,3cyclictest964968-21latency_hist19:45:001
928448993935,3cyclictest1120393-21gpgv22:14:590
928458993835,2cyclictest1136267-21kworker/u8:2+events_unbound22:40:233
928453993835,2cyclictest1053110-21kworker/u8:3+flush-259:022:10:242
928448993833,3cyclictest980975-21latency_hist20:00:000
928458993734,2cyclictest1193647-21kworker/u8:2+flush-259:023:30:223
928458993734,2cyclictest1136267-21kworker/u8:2+events_unbound22:45:183
928450993733,3cyclictest1053110-21kworker/u8:3+flush-259:022:05:001
928458993633,2cyclictest996528-21kworker/u8:2+events_unbound20:45:143
928458993633,2cyclictest1240430-21kworker/u8:0+flush-259:000:35:203
928458993633,2cyclictest1173268-21kworker/u8:4+flush-259:000:20:253
928458993632,3cyclictest869868-21kworker/u8:1+events_unbound19:30:173
928448993630,4cyclictest1264962-21ntpq00:30:200
928458993532,2cyclictest975805-21kworker/u8:1+events_unbound19:55:173
928458993532,2cyclictest1240430-21kworker/u8:0+events_unbound00:30:243
928458993532,2cyclictest1173268-21kworker/u8:4+events_unbound23:40:203
928458993527,2cyclictest1173268-21kworker/u8:4+events_unbound00:00:153
928453993532,2cyclictest1027365-21kworker/u8:1+flush-259:021:10:232
928448993530,3cyclictest965567-21base6419:45:010
928458993431,2cyclictest869868-21kworker/u8:1+flush-259:019:10:213
928458993431,2cyclictest1219688-21kworker/u8:1+flush-259:023:50:243
928453993431,2cyclictest1073899-21kworker/u8:0+flush-259:022:00:202
928453993431,2cyclictest1027365-21kworker/u8:1+flush-259:021:00:252
928450993430,2cyclictest1142073-21kworker/u8:0+flush-259:023:40:011
928458993329,3cyclictest1173268-21kworker/u8:4+events_unbound23:35:183
928458993329,3cyclictest1142073-21kworker/u8:0+events_unbound22:55:213
928458993328,4cyclictest965626-21/usr/sbin/munin19:45:143
928453993330,2cyclictest1136267-21kworker/u8:2+events_unbound22:50:202
928453993329,3cyclictest1147271-21apt-get22:40:132
928450993329,2cyclictest869868-21kworker/u8:1+flush-259:019:10:151
928458993229,2cyclictest1142073-21kworker/u8:0+flush-259:023:00:213
928458993229,2cyclictest1073899-21kworker/u8:0+flush-259:021:55:153
928458993229,2cyclictest1027365-21kworker/u8:1+events_unbound21:30:223
928453993229,2cyclictest1053110-21kworker/u8:3+flush-259:021:40:212
928448993226,4cyclictest1229913-21latency_hist00:00:010
928458993128,3cyclictest1053110-21kworker/u8:3+flush-259:021:45:163
928458993128,2cyclictest996528-21kworker/u8:2+flush-259:020:55:223
928458993128,2cyclictest938469-21kworker/u8:2+events_unbound19:25:143
928458993128,2cyclictest1130883-21kworker/u8:3+events_unbound22:30:223
928453993128,2cyclictest965617-21kworker/u8:4+flush-259:020:00:202
928453993128,2cyclictest1053110-21kworker/u8:3+flush-259:021:50:162
928453993126,3cyclictest938469-21kworker/u8:2+flush-259:019:25:172
928448993126,4cyclictest1036764-21ntpq20:50:230
928448993125,4cyclictest335557-21systemd-journal21:58:290
928458993026,3cyclictest1110088-21kworker/u8:2+events_unbound22:15:013
928453993027,2cyclictest1006831-21kworker/u8:3+events_unbound20:35:222
928453993026,3cyclictest1130883-21kworker/u8:3+events_unbound22:30:272
928450993028,1cyclictest1073899-21kworker/u8:0+events_unbound22:05:141
928450993027,2cyclictest975805-21kworker/u8:1+events_unbound20:30:151
928448993027,2cyclictest1173268-21kworker/u8:4+events_unbound00:00:200
928458992926,2cyclictest918350-21kworker/u8:0+events_unbound19:15:203
928458992926,2cyclictest1173268-21kworker/u8:4+events_unbound23:55:243
928453992926,2cyclictest965617-21kworker/u8:4+flush-259:019:55:202
928453992926,2cyclictest1173268-21kworker/u8:4+flush-259:023:10:162
928453992926,2cyclictest1110088-21kworker/u8:2+events_unbound22:15:172
928453992926,2cyclictest1027365-21kworker/u8:1+events_unbound20:50:192
928453992925,3cyclictest1053110-21kworker/u8:3+events_unbound21:45:212
928450992925,3cyclictest1168344-21apt-get23:00:111
928448992925,3cyclictest996528-21kworker/u8:2+flush-259:021:05:210
928458992825,2cyclictest996528-21kworker/u8:2+events_unbound20:40:123
928458992825,2cyclictest1240430-21kworker/u8:0+events_unbound00:15:143
928458992825,2cyclictest1173268-21kworker/u8:4+flush-259:023:10:193
928453992826,2cyclictest1142073-21kworker/u8:0+flush-259:023:05:202
928453992825,2cyclictest1053110-21kworker/u8:3+flush-259:021:35:202
928453992824,3cyclictest1136259-21gpgv22:30:002
928450992825,2cyclictest1193647-21kworker/u8:2+events_unbound23:25:221
928448992823,3cyclictest335557-21systemd-journal23:37:390
928458992724,2cyclictest869868-21kworker/u8:1+events_unbound19:40:183
928453992724,2cyclictest869868-21kworker/u8:1+flush-259:019:30:212
928453992724,2cyclictest1219688-21kworker/u8:1+flush-259:000:05:212
928453992724,2cyclictest1110088-21kworker/u8:2+events_unbound22:20:142
928450992724,2cyclictest1240430-21kworker/u8:0+events_unbound00:35:221
928450992723,3cyclictest1073899-21kworker/u8:0+events_unbound22:21:301
928458992623,2cyclictest996528-21kworker/u8:2+events_unbound21:05:223
928458992623,2cyclictest1006831-21kworker/u8:3+flush-259:020:25:203
928453992623,2cyclictest1130883-21kworker/u8:3+events_unbound23:00:222
928453992621,4cyclictest1073664-21latency_hist21:30:012
928450992623,2cyclictest959370-21kworker/u8:0+events_unbound19:50:261
928450992623,2cyclictest1130883-21kworker/u8:3+flush-259:023:15:201
928450992618,3cyclictest996528-21kworker/u8:2+events_unbound20:15:221
928448992622,3cyclictest918350-21kworker/u8:0+events_unbound19:15:220
928458992522,2cyclictest1142073-21kworker/u8:0+flush-259:023:45:193
928458992522,2cyclictest1136267-21kworker/u8:2+events_unbound22:35:163
928458992522,2cyclictest1136267-21kworker/u8:2+events_unbound22:35:153
928458992522,2cyclictest1130883-21kworker/u8:3+flush-259:022:25:203
928458992521,3cyclictest938469-21kworker/u8:2+events_unbound19:35:133
928458992521,3cyclictest1240461-21apt-get00:10:103
928458992521,3cyclictest1188690-21latency_hist23:20:003
928453992522,2cyclictest1255877-21kworker/u8:1+events_unbound00:25:172
928453992522,2cyclictest1193647-21kworker/u8:2+flush-259:023:25:212
928453992522,2cyclictest1173268-21kworker/u8:4+events_unbound00:20:212
928453992522,2cyclictest1142073-21kworker/u8:0+events_unbound22:55:172
928453992522,2cyclictest1053110-21kworker/u8:3+events_unbound21:20:222
928450992522,2cyclictest1173268-21kworker/u8:4+flush-259:023:55:011
928450992522,2cyclictest1152194-21kworker/u8:4+flush-259:022:50:001
928450992522,2cyclictest1142073-21kworker/u8:0+events_unbound23:05:161
928450992522,2cyclictest1027365-21kworker/u8:1+events_unbound21:30:201
928448992522,2cyclictest1142073-21kworker/u8:0+events_unbound23:45:250
928458992421,2cyclictest1073899-21kworker/u8:0+flush-259:022:20:213
928458992420,3cyclictest1255877-21kworker/u8:1+events_unbound00:25:303
928458992418,4cyclictest928384-21latency_hist19:10:013
928453992421,2cyclictest996528-21kworker/u8:2+flush-259:020:40:172
928450992421,2cyclictest1240430-21kworker/u8:0+flush-259:000:10:151
928450992421,2cyclictest1053110-21kworker/u8:3+events_unbound21:15:251
928450992420,3cyclictest1053110-21kworker/u8:3+flush-259:021:50:211
928448992421,2cyclictest1027365-21kworker/u8:1+events_unbound21:25:230
928448992420,3cyclictest1173268-21kworker/u8:4+flush-259:023:40:250
928448992420,3cyclictest1173268-21kworker/u8:4+events_unbound00:20:160
928448992420,3cyclictest1073899-21kworker/u8:0+events_unbound21:50:230
928448992419,3cyclictest335557-21systemd-journal23:50:000
92844899240,22cyclictest335557-21systemd-journal22:15:010
928458992321,2cyclictest975805-21kworker/u8:1+events_unbound20:05:213
928458992320,2cyclictest1136267-21kworker/u8:2+flush-259:022:50:253
928458992320,2cyclictest1053110-21kworker/u8:3+flush-259:021:40:223
928453992320,2cyclictest975805-21kworker/u8:1+flush-259:020:05:252
928453992320,2cyclictest975805-21kworker/u8:1+events_unbound20:30:212
928453992320,2cyclictest959370-21kworker/u8:0+flush-259:019:45:242
928453992318,4cyclictest1011972-21latency_hist20:30:012
928450992320,2cyclictest918350-21kworker/u8:0+events_unbound19:15:151
928450992320,2cyclictest1240430-21kworker/u8:0+events_unbound00:25:211
928450992320,2cyclictest1136267-21kworker/u8:2+events_unbound22:40:161
928450992319,3cyclictest975805-21kworker/u8:1+events_unbound19:55:141
928450992319,3cyclictest938469-21kworker/u8:2+flush-259:019:25:231
928450992319,3cyclictest1136896-21/usr/sbin/munin22:30:151
928450992318,4cyclictest1053110-21kworker/u8:3+events_unbound21:57:271
928448992320,2cyclictest1173268-21kworker/u8:4+events_unbound23:20:190
928448992319,3cyclictest1130883-21kworker/u8:3+events_unbound23:15:230
928448992319,3cyclictest1053110-21kworker/u8:3+events_unbound21:20:230
928448992318,4cyclictest1142073-21kworker/u8:0+flush-259:023:15:000
928458992218,3cyclictest1027365-21kworker/u8:1+flush-259:021:10:123
928453992220,2cyclictest1240430-21kworker/u8:0+events_unbound00:10:202
928453992220,2cyclictest1073899-21kworker/u8:0+flush-259:021:55:192
928453992219,2cyclictest938469-21kworker/u8:2+flush-259:019:35:182
928453992219,2cyclictest1219688-21kworker/u8:1+events_unbound23:50:232
928453992218,3cyclictest509-21rs:main2
928453992218,3cyclictest1240430-21kworker/u8:0+events_unbound00:30:282
928450992219,2cyclictest975805-21kworker/u8:1+events_unbound20:05:191
928450992219,2cyclictest1173268-21kworker/u8:4+flush-259:023:55:201
928450992219,2cyclictest1142073-21kworker/u8:0+events_unbound23:45:231
928450992219,2cyclictest1027365-21kworker/u8:1+events_unbound21:25:151
928450992218,3cyclictest1173268-21kworker/u8:4+events_unbound00:19:311
928450992218,3cyclictest1142073-21kworker/u8:0+flush-259:022:35:301
928450992218,3cyclictest1142073-21kworker/u8:0+flush-259:022:35:301
928450992218,3cyclictest1120236-21kworker/u8:1+events_unbound22:26:301
928450992218,3cyclictest1053110-21kworker/u8:3+events_unbound21:45:231
928450992218,3cyclictest1017120-21kworker/u8:0+events_unbound20:36:301
928448992218,3cyclictest996528-21kworker/u8:2+flush-259:020:55:250
928448992218,3cyclictest975805-21kworker/u8:1+flush-259:020:20:190
928448992218,3cyclictest1073899-21kworker/u8:0+flush-259:022:00:160
928448992215,5cyclictest1167219-21ntpq22:55:200
928448992215,5cyclictest1016256-21ntpq20:30:220
928458992118,2cyclictest1142073-21kworker/u8:0+flush-259:023:05:183
928458992118,2cyclictest1053110-21kworker/u8:3+events_unbound22:10:113
928458992116,3cyclictest970998-21diskstats19:50:133
928450992118,2cyclictest959370-21kworker/u8:0+events_unbound20:10:221
928450992118,2cyclictest1136267-21kworker/u8:2+flush-259:023:00:011
928450992118,2cyclictest1027365-21kworker/u8:1+events_unbound21:10:141
928450992117,3cyclictest996528-21kworker/u8:2+events_unbound20:55:231
928450992117,3cyclictest975805-21kworker/u8:1+events_unbound20:20:211
928450992117,3cyclictest965617-21kworker/u8:4+flush-259:019:45:161
928450992117,3cyclictest1173268-21kworker/u8:4+events_unbound23:11:301
928450992115,4cyclictest335557-21systemd-journal21:00:291
928450992114,5cyclictest1254762-21ntpq00:20:211
928448992117,3cyclictest996528-21kworker/u8:2+events_unbound20:15:160
928448992117,3cyclictest1193647-21kworker/u8:2+events_unbound23:28:300
928448992116,4cyclictest985190-21ntpq20:00:210
928448992116,3cyclictest509-21rs:main0
928458992017,2cyclictest1027365-21kworker/u8:1+events_unbound20:50:133
928453992018,2cyclictest959370-21kworker/u8:0+events_unbound20:10:262
928453992018,1cyclictest943914-21kworker/u8:3+events_unbound19:50:122
928453992017,2cyclictest975805-21kworker/u8:1+flush-259:020:20:242
928453992016,3cyclictest1173268-21kworker/u8:4+events_unbound23:18:292
928450992017,2cyclictest938469-21kworker/u8:2+events_unbound19:36:301
928450992017,2cyclictest1136267-21kworker/u8:2+events_unbound22:50:311
928450992017,2cyclictest1027365-21kworker/u8:1+events_unbound20:50:211
928450992016,3cyclictest975805-21kworker/u8:1+events_unbound20:02:121
928448992017,2cyclictest869868-21kworker/u8:1+events_unbound19:31:290
928448992017,2cyclictest1053110-21kworker/u8:3+flush-259:021:45:210
928448992016,3cyclictest1142073-21kworker/u8:0+flush-259:022:35:160
928448992016,3cyclictest1142073-21kworker/u8:0+flush-259:022:35:150
928448992016,3cyclictest1006831-21kworker/u8:3+events_unbound20:25:190
928458991916,2cyclictest959370-21kworker/u8:0+events_unbound20:10:243
928458991916,2cyclictest938469-21kworker/u8:2+events_unbound19:21:333
928453991916,2cyclictest1240430-21kworker/u8:0+events_unbound00:35:122
928453991916,2cyclictest1130883-21kworker/u8:3+flush-259:023:20:212
928453991915,3cyclictest1173268-21kworker/u8:4+events_unbound23:42:412
928453991914,4cyclictest335557-21systemd-journal21:35:002
928450991916,2cyclictest1053110-21kworker/u8:3+flush-259:021:35:271
928450991915,3cyclictest1255877-21kworker/u8:1+events_unbound00:32:311
928450991913,5cyclictest1213621-21ntpq23:40:201
928448991915,3cyclictest1240430-21kworker/u8:0+events_unbound00:35:130
928448991915,3cyclictest1130883-21kworker/u8:3+events_unbound23:04:300
928448991915,3cyclictest1130883-21kworker/u8:3+events_unbound22:45:270
928448991915,3cyclictest1073899-21kworker/u8:0+events_unbound22:05:120
928448991914,4cyclictest990457-21ntpq20:05:220
928448991913,4cyclictest1162515-21latency_hist22:55:010
928448991913,4cyclictest1062541-21ntpq21:15:230
928458991815,2cyclictest975805-21kworker/u8:1+flush-259:020:30:183
928458991815,2cyclictest1006831-21kworker/u8:3+flush-259:020:35:253
928458991814,3cyclictest996528-21kworker/u8:2+flush-259:021:05:003
928453991816,2cyclictest996528-21kworker/u8:2+events_unbound20:15:262
928453991815,2cyclictest869868-21kworker/u8:1+flush-259:019:20:262
928453991815,2cyclictest3042295-21kworker/u8:3+events_unbound19:12:302
928453991814,3cyclictest996528-21kworker/u8:2+flush-259:020:45:192
928453991814,3cyclictest1173268-21kworker/u8:4+events_unbound00:02:572
928450991815,2cyclictest996528-21kworker/u8:2+flush-259:020:45:181
928450991815,2cyclictest996528-21kworker/u8:2+events_unbound20:26:341
928450991815,2cyclictest1110088-21kworker/u8:2+flush-259:022:10:301
928450991814,3cyclictest869868-21kworker/u8:1+events_unbound19:20:121
928450991814,3cyclictest1229946-21kworker/u8:2+flush-259:000:00:311
928450991813,4cyclictest1204192-21latency_hist23:35:011
928448991815,2cyclictest869868-21kworker/u8:1+events_unbound19:40:250
928448991815,2cyclictest869868-21kworker/u8:1+events_unbound19:20:300
928448991814,3cyclictest996528-21kworker/u8:2+events_unbound21:00:140
928448991814,3cyclictest1136267-21kworker/u8:2+events_unbound22:42:300
928448991813,4cyclictest948255-21ntpq19:25:230
928458991714,2cyclictest975805-21kworker/u8:1+events_unbound20:21:513
928458991714,2cyclictest1053110-21kworker/u8:3+events_unbound21:15:273
928458991713,3cyclictest1142073-21kworker/u8:0+events_unbound23:26:403
928453991714,2cyclictest1136267-21kworker/u8:2+flush-259:022:45:202
928453991713,3cyclictest1173268-21kworker/u8:4+events_unbound23:36:372
928450991713,3cyclictest1073899-21kworker/u8:0+events_unbound22:19:221
928448991714,2cyclictest60350irq/132-enp0s29f1:rx-023:07:000
928448991714,2cyclictest507-21in:imuxsock21:35:010
928448991713,3cyclictest1255877-21kworker/u8:1+flush-259:000:25:150
928448991713,3cyclictest1173268-21kworker/u8:4+events_unbound00:11:310
928448991712,3cyclictest335557-21systemd-journal21:15:000
928458991613,2cyclictest1053110-21kworker/u8:3+events_unbound21:29:143
928453991614,2cyclictest938469-21kworker/u8:2+events_unbound19:19:222
928453991613,2cyclictest1027365-21kworker/u8:1+events_unbound21:19:212
928453991613,2cyclictest1027365-21kworker/u8:1+events_unbound21:09:282
928450991612,3cyclictest1173268-21kworker/u8:4+events_unbound23:20:211
928450991612,3cyclictest1027365-21kworker/u8:1+events_unbound21:10:011
928448991614,1cyclictest60350irq/132-enp0s29f1:rx-022:26:000
928448991614,1cyclictest60350irq/132-enp0s29f1:rx-021:35:130
928448991613,2cyclictest1027365-21kworker/u8:1+events_unbound20:49:300
928448991612,3cyclictest959370-21kworker/u8:0+flush-259:020:10:230
928448991612,3cyclictest928663-21apt-get19:10:110
928448991612,3cyclictest1240430-21kworker/u8:0+events_unbound00:15:230
92845899159,5cyclictest335557-21systemd-journal21:50:033
928458991511,3cyclictest1053110-21kworker/u8:3+events_unbound21:23:333
928453991512,2cyclictest1240430-21kworker/u8:0+events_unbound00:15:332
92845099159,1cyclictest0-21swapper/120:45:001
928450991511,3cyclictest335557-21systemd-journal21:25:011
928448991513,1cyclictest60350irq/132-enp0s29f1:rx-022:32:340
928448991513,1cyclictest60350irq/132-enp0s29f1:rx-000:06:410
928448991512,2cyclictest1073899-21kworker/u8:0+events_unbound22:20:190
928448991510,4cyclictest973404-21cut19:50:180
928453991410,3cyclictest996528-21kworker/u8:2+events_unbound20:59:022
928450991411,2cyclictest1219688-21kworker/u8:1+events_unbound00:05:261
928448991411,2cyclictest60350irq/132-enp0s29f1:rx-023:30:230
928448991411,2cyclictest60350irq/132-enp0s29f1:rx-021:40:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional