You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-08 - 23:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Sat Nov 08, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26931211693,16sleep30-21swapper/307:05:233
27499211597,12sleep20-21swapper/207:07:332
273882113102,6sleep00-21swapper/007:06:070
27349210180,14sleep10-21swapper/107:05:361
187382770,1sleep337-21rcuc/310:55:273
315602730,0sleep10-21swapper/110:02:301
283992700,0sleep00-21swapper/010:32:270
224552570,1sleep20-21swapper/209:20:142
27718993617,18cyclictest221ktimersoftd/109:25:121
180912280,0sleep20-21swapper/209:49:202
281092260,1sleep00-21swapper/012:12:080
320862220,1sleep30-21swapper/308:40:003
326912210,0sleep1221ktimersoftd/108:40:141
129632210,0sleep30-21swapper/310:16:073
2772099201,2cyclictest0-21swapper/309:25:113
2772099200,18cyclictest0-21swapper/311:40:123
25942200,0sleep20-21swapper/209:33:142
2772099181,14cyclictest0-21swapper/309:40:183
27720991712,3cyclictest187892chrt09:15:423
2772099170,15cyclictest30547-21tr12:15:123
23342170,1sleep22330-21ssh12:20:002
27720991611,3cyclictest2959-21cpuspeed_turbos12:20:143
2772099161,13cyclictest18525-21apt-get09:50:063
27720991611,3cyclictest15160-21ssh12:32:123
2771999161,13cyclictest5129-21runrttasks11:22:112
2771999160,14cyclictest0-21swapper/211:40:122
2771899168,7cyclictest0-21swapper/109:40:581
27718991612,2cyclictest0-21swapper/108:45:131
2772099155,7cyclictest5911-21gdm-simple-gree10:03:583
2772099151,12cyclictest10110-21iostat_ios11:20:183
27720991510,3cyclictest7727-21ssh11:50:293
2772099150,3cyclictest0-21swapper/310:44:383
2772099150,2cyclictest0-21swapper/311:59:323
2772099150,2cyclictest0-21swapper/309:35:223
2772099150,2cyclictest0-21swapper/307:35:123
2772099150,13cyclictest501-21ssh10:37:023
2772099150,13cyclictest0-21swapper/311:08:183
2771999151,12cyclictest28662-21apt-get10:00:122
2771999151,12cyclictest0-21swapper/210:20:152
2771999150,13cyclictest0-21swapper/208:40:122
2771899156,5cyclictest0-21swapper/111:08:581
27718991510,3cyclictest0-21swapper/112:00:011
27718991510,3cyclictest0-21swapper/109:37:251
2771899150,2cyclictest11896-21diskmemload12:33:521
2771899150,13cyclictest24924-21munin-node1
2771899150,11cyclictest0-21swapper/111:32:161
2771799150,13cyclictest0-21swapper/010:44:590
2772099142,7cyclictest0-21swapper/308:30:003
27720991411,2cyclictest0-21swapper/311:33:123
27720991411,2cyclictest0-21swapper/309:49:423
27720991411,2cyclictest0-21swapper/308:05:123
2772099140,3cyclictest15592-21ssh10:52:363
2772099140,13cyclictest0-21swapper/312:05:223
2772099140,13cyclictest0-21swapper/310:46:523
2772099140,12cyclictest7412-21unixbench_multi08:55:283
2772099140,12cyclictest0-21swapper/311:18:263
2772099140,12cyclictest0-21swapper/310:20:263
2772099140,12cyclictest0-21swapper/307:20:253
2772099140,12cyclictest0-21swapper/307:15:073
27719991411,2cyclictest7851-21apt-get07:40:122
2771999140,3cyclictest0-21swapper/211:37:222
2771999140,2cyclictest0-21swapper/209:38:202
2771999140,12cyclictest0-21swapper/211:25:272
2771999140,12cyclictest0-21swapper/209:40:262
2771899149,4cyclictest0-21swapper/110:05:511
2771899141,2cyclictest31090-21iostat_ios10:35:181
2771899141,11cyclictest9396-21apt-get11:20:131
2771899141,11cyclictest8849-21cut07:40:191
27718991410,2cyclictest21638-21ntp_states10:25:201
27718991410,2cyclictest0-21swapper/111:10:081
2771899140,2cyclictest0-21swapper/110:20:411
2771899140,13cyclictest0-21swapper/109:14:341
2771899140,12cyclictest29976-21swap11:40:261
2771899140,12cyclictest11896-21diskmemload09:32:011
2771899140,12cyclictest0-21swapper/111:50:061
2771899140,12cyclictest0-21swapper/111:45:121
2771899140,12cyclictest0-21swapper/109:50:461
27717991411,2cyclictest5936-21kworker/u8:212:23:040
27717991411,2cyclictest15982-21kworker/u8:109:52:040
27717991411,2cyclictest0-21swapper/012:06:090
27717991411,2cyclictest0-21swapper/009:44:190
2771799140,2cyclictest12260-21cat10:15:260
2771799140,2cyclictest0-21swapper/011:39:390
2771799140,2cyclictest0-21swapper/011:32:320
2771799140,13cyclictest0-21swapper/011:11:090
2771799140,12cyclictest12658-21ssh11:55:380
2771799140,12cyclictest11896-21diskmemload09:37:070
2771799140,12cyclictest11896-21diskmemload09:15:170
2771799140,12cyclictest0-21swapper/012:30:200
2771799140,12cyclictest0-21swapper/009:30:260
2772099132,6cyclictest3704-21copy08:50:013
2772099132,6cyclictest0-21swapper/307:55:013
2772099132,6cyclictest0-21swapper/307:55:003
2772099131,7cyclictest19946-21copy08:10:013
27720991310,2cyclictest11896-21diskmemload12:26:443
27720991310,2cyclictest11896-21diskmemload11:25:143
27720991310,2cyclictest0-21swapper/312:00:433
27720991310,2cyclictest0-21swapper/310:31:273
27720991310,2cyclictest0-21swapper/309:55:243
2772099130,2cyclictest0-21swapper/311:36:243
2772099130,2cyclictest0-21swapper/311:12:423
2772099130,2cyclictest0-21swapper/311:03:153
2772099130,12cyclictest0-21swapper/312:36:133
2772099130,12cyclictest0-21swapper/312:11:053
2772099130,12cyclictest0-21swapper/310:27:123
27719991310,2cyclictest3125-21kworker/u8:009:50:042
27719991310,2cyclictest25606-21kworker/u8:007:12:042
27719991310,2cyclictest19894-21turbostat.cron08:10:012
27719991310,2cyclictest0-21swapper/212:29:452
27719991310,2cyclictest0-21swapper/212:20:212
27719991310,2cyclictest0-21swapper/211:34:302
27719991310,2cyclictest0-21swapper/211:19:212
27719991310,2cyclictest0-21swapper/210:44:522
27719991310,2cyclictest0-21swapper/210:31:452
27719991310,2cyclictest0-21swapper/210:18:422
27719991310,2cyclictest0-21swapper/210:05:172
27719991310,2cyclictest0-21swapper/209:10:002
27719991310,2cyclictest0-21swapper/208:25:202
2771999130,2cyclictest21596-21ssh12:05:182
2771999130,2cyclictest0-21swapper/210:59:072
2771999130,12cyclictest3908-21ssh11:47:212
2771999130,12cyclictest20686-21apt-get10:25:132
2771999130,12cyclictest0-21swapper/212:32:402
2771999130,12cyclictest0-21swapper/212:11:412
2771999130,12cyclictest0-21swapper/211:50:192
2771999130,12cyclictest0-21swapper/211:08:312
2771999130,12cyclictest0-21swapper/209:58:132
2771999130,12cyclictest0-21swapper/208:15:142
27718991310,2cyclictest0-21swapper/112:09:001
27718991310,2cyclictest0-21swapper/111:28:341
27718991310,2cyclictest0-21swapper/111:15:581
27718991310,2cyclictest0-21swapper/110:32:371
27718991310,2cyclictest0-21swapper/110:10:461
27718991310,2cyclictest0-21swapper/109:19:281
27718991310,2cyclictest0-21swapper/108:10:201
2771899130,3cyclictest0-21swapper/108:20:211
2771899130,2cyclictest0-21swapper/112:14:491
2771899130,2cyclictest0-21swapper/112:00:491
2771899130,2cyclictest0-21swapper/109:45:281
2771899130,2cyclictest0-21swapper/107:25:131
2771899130,12cyclictest0-21swapper/112:35:201
2771899130,12cyclictest0-21swapper/112:28:521
2771899130,12cyclictest0-21swapper/108:30:021
2771899130,11cyclictest11896-21diskmemload10:46:331
2771899130,11cyclictest0-21swapper/112:20:051
2771899130,11cyclictest0-21swapper/111:39:151
2771799131,2cyclictest0-21swapper/010:35:340
27717991310,2cyclictest28840-21kworker/u8:110:02:040
27717991310,2cyclictest0-21swapper/012:37:190
27717991310,2cyclictest0-21swapper/012:15:260
27717991310,2cyclictest0-21swapper/011:41:250
27717991310,2cyclictest0-21swapper/010:46:120
27717991310,2cyclictest0-21swapper/009:56:170
27717991310,2cyclictest0-21swapper/009:46:140
27717991310,2cyclictest0-21swapper/009:25:160
27717991310,2cyclictest0-21swapper/008:50:140
27717991310,2cyclictest0-21swapper/008:40:170
27717991310,2cyclictest0-21swapper/008:10:270
27717991310,2cyclictest0-21swapper/007:35:010
27717991310,2cyclictest0-21swapper/007:25:240
2771799130,2cyclictest22308-21ssh11:00:090
2771799130,2cyclictest0-21swapper/011:49:290
2771799130,2cyclictest0-21swapper/011:15:400
2771799130,12cyclictest7614-21ssh12:25:100
2771799130,12cyclictest0-21swapper/012:00:090
2771799130,12cyclictest0-21swapper/011:50:350
2771799130,12cyclictest0-21swapper/011:05:180
2771799130,12cyclictest0-21swapper/010:55:280
2771799130,12cyclictest0-21swapper/010:50:090
2771799130,12cyclictest0-21swapper/010:10:140
2771799130,12cyclictest0-21swapper/009:20:380
2771799130,12cyclictest0-21swapper/008:55:170
2771799130,12cyclictest0-21swapper/008:00:260
2771799130,12cyclictest0-21swapper/007:45:000
2771799130,11cyclictest0-21swapper/011:26:580
15352130,2sleep20-21swapper/210:38:392
2772099129,2cyclictest0-21swapper/310:07:383
2772099129,2cyclictest0-21swapper/309:20:063
2772099129,2cyclictest0-21swapper/309:10:073
2772099129,2cyclictest0-21swapper/309:05:173
2772099129,2cyclictest0-21swapper/308:40:253
2772099129,2cyclictest0-21swapper/307:10:233
2772099122,7cyclictest0-21swapper/309:04:013
2772099120,11cyclictest0-21swapper/311:45:183
2772099120,11cyclictest0-21swapper/310:10:103
2772099120,11cyclictest0-21swapper/309:31:563
2772099120,11cyclictest0-21swapper/308:30:263
2771999129,2cyclictest0-21swapper/211:11:222
2771999129,2cyclictest0-21swapper/211:02:502
2771999129,2cyclictest0-21swapper/210:50:322
2771999129,2cyclictest0-21swapper/210:48:162
2771999129,2cyclictest0-21swapper/207:35:262
2771999121,2cyclictest0-21swapper/209:26:182
2771999120,2cyclictest0-21swapper/212:35:042
2771999120,2cyclictest0-21swapper/211:55:272
2771999120,11cyclictest0-21swapper/212:03:022
2771999120,11cyclictest0-21swapper/209:15:292
2771999120,11cyclictest0-21swapper/209:00:162
2771999120,11cyclictest0-21swapper/207:50:172
2771999120,11cyclictest0-21swapper/207:50:172
2771999120,11cyclictest0-21swapper/207:15:112
2771899129,2cyclictest0-21swapper/112:16:151
2771899129,2cyclictest0-21swapper/110:40:191
2771899129,2cyclictest0-21swapper/110:17:061
2771899129,2cyclictest0-21swapper/109:22:151
2771899129,2cyclictest0-21swapper/109:05:191
2771899129,2cyclictest0-21swapper/108:55:211
2771899129,2cyclictest0-21swapper/108:35:151
2771899123,6cyclictest11896-21diskmemload10:55:271
2771899121,9cyclictest3635-21apt-get08:50:051
2771899120,2cyclictest0-21swapper/111:00:121
2771899120,11cyclictest0-21swapper/110:50:091
2771899120,11cyclictest0-21swapper/109:00:161
2771899120,11cyclictest0-21swapper/108:25:181
2771899120,11cyclictest0-21swapper/108:05:141
2771899120,11cyclictest0-21swapper/107:50:171
2771899120,11cyclictest0-21swapper/107:50:171
2771899120,11cyclictest0-21swapper/107:35:131
2771799129,2cyclictest0-21swapper/010:05:100
2771799129,2cyclictest0-21swapper/007:40:190
2771799120,2cyclictest0-21swapper/011:20:200
2771799120,2cyclictest0-21swapper/007:10:160
2771799120,11cyclictest11896-21diskmemload09:10:460
2771799120,11cyclictest0-21swapper/010:25:100
2771799120,11cyclictest0-21swapper/010:22:010
2771799120,11cyclictest0-21swapper/008:30:180
118252121,8sleep20-21swapper/207:50:012
2772099118,2cyclictest0-21swapper/307:25:273
2772099111,6cyclictest0-21swapper/308:50:123
2771999118,2cyclictest31451-21kworker/u8:207:23:052
2771999118,2cyclictest19807-21kworker/u8:208:13:042
2771999118,2cyclictest13725-21kworker/u8:107:58:042
2771999118,2cyclictest0-21swapper/210:12:422
2771999118,2cyclictest0-21swapper/207:25:192
2771899118,2cyclictest0-21swapper/107:20:261
2771899110,3cyclictest0-21swapper/107:10:171
2771799118,2cyclictest0-21swapper/008:45:250
2772099108,1cyclictest0-21swapper/307:55:253
2772099105,3cyclictest9634-21cron07:45:003
2772099105,3cyclictest10257-21df_abs07:45:153
2772099104,4cyclictest4166-21rsyslogd08:05:003
2772099102,6cyclictest0-21swapper/308:15:263
2772099100,2cyclictest0-21swapper/307:35:013
2771999107,2cyclictest5596-21sed07:35:002
2771899107,2cyclictest0-21swapper/107:30:301
2771899102,5cyclictest0-21swapper/107:58:011
2771899100,9cyclictest0-21swapper/108:15:231
2771899100,8cyclictest10767-21apt-get07:45:251
2771799107,2cyclictest0-21swapper/007:20:110
2771799106,2cyclictest11125-21pzem_voltage09:05:220
2771799104,2cyclictest0-21swapper/007:19:040
277209993,3cyclictest24311-21telnet08:20:133
277199996,2cyclictest26222-21kworker/u8:008:38:042
277199994,3cyclictest5129-21runrttasks08:52:452
277199994,3cyclictest4809-21gdbus08:55:102
277199994,3cyclictest3158-21tail08:45:272
277199994,3cyclictest10162-21head09:05:122
277189991,7cyclictest0-21swapper/108:00:241
277189990,2cyclictest0-21swapper/107:15:051
277179996,2cyclictest0-21swapper/008:15:200
277179996,2cyclictest0-21swapper/008:09:000
277179994,3cyclictest12303-21cpuspeed_turbos07:50:150
277179994,3cyclictest12303-21cpuspeed_turbos07:50:150
277179993,2cyclictest0-21swapper/009:03:530
277179990,8cyclictest0-21swapper/007:57:010
17792290,2sleep20-21swapper/208:03:592
277199983,3cyclictest4508-21dbus-daemon08:20:042
277199983,3cyclictest28564-21df_abs08:30:162
277179985,2cyclictest0-21swapper/008:35:200
277179985,2cyclictest0-21swapper/007:30:190
277179980,2cyclictest0-21swapper/008:27:240
277179973,3cyclictest4324-21nscd08:20:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional