You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-02 - 23:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Thu Jul 02, 2026 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25489992523,1cyclictest0-21swapper/311:26:143
2548899250,24cyclictest4987-21cron07:35:002
2548899250,1cyclictest27518-21sh10:14:422
25487992523,1cyclictest0-21swapper/112:39:431
2548799250,25cyclictest0-21swapper/111:39:101
2548799250,25cyclictest0-21swapper/110:43:291
2548299252,22cyclictest62550irq/127-eno111:47:030
25482992522,2cyclictest5284-21awk09:55:000
25489992422,1cyclictest0-21swapper/310:00:103
2548999240,24cyclictest0-21swapper/310:58:423
2548999240,23cyclictest5316-21munin-run09:55:003
2548999240,1cyclictest0-21swapper/310:44:303
25488992421,2cyclictest13255-21ssh09:32:452
25488992421,1cyclictest0-21swapper/210:55:032
2548899240,24cyclictest29513-21ssh11:13:142
2548899240,23cyclictest19208-21diskmemload10:01:252
2548899240,0cyclictest0-21swapper/210:49:402
25487992422,1cyclictest0-21swapper/112:27:561
25482992423,1cyclictest62550irq/127-eno111:05:570
25482992421,2cyclictest0-21swapper/010:06:190
2548299240,24cyclictest0-21swapper/012:02:530
2548299240,24cyclictest0-21swapper/011:27:570
2548299240,23cyclictest0-21swapper/010:03:080
2548299240,1cyclictest0-21swapper/011:34:470
2548299240,1cyclictest0-21swapper/010:28:370
25489992323,0cyclictest0-21swapper/311:54:383
25489992323,0cyclictest0-21swapper/310:48:243
25489992323,0cyclictest0-21swapper/310:27:033
25489992323,0cyclictest0-21swapper/309:15:533
25489992322,1cyclictest22256-21users09:11:203
25489992322,1cyclictest19208-21diskmemload12:19:123
25489992322,1cyclictest19208-21diskmemload10:31:383
25489992321,1cyclictest0-21swapper/312:35:083
25489992321,1cyclictest0-21swapper/312:06:443
2548999231,21cyclictest22355-21apt08:10:003
2548999231,1cyclictest0-21swapper/310:39:553
2548999230,2cyclictest24659-21ptp4l-jitter10:11:173
2548999230,23cyclictest0-21swapper/312:30:453
2548999230,23cyclictest0-21swapper/312:30:453
2548999230,23cyclictest0-21swapper/312:22:483
2548999230,23cyclictest0-21swapper/311:22:303
2548999230,23cyclictest0-21swapper/309:59:513
2548999230,22cyclictest19792-21ssh12:01:393
2548999230,22cyclictest19208-21diskmemload11:10:493
2548999230,22cyclictest15301-21sh09:35:083
2548999230,22cyclictest12364-21apt-get08:56:103
2548999230,22cyclictest10372-21sh07:45:003
2548999230,22cyclictest0-21swapper/311:45:413
2548999230,21cyclictest17367-21diskmemload10:05:493
2548999230,1cyclictest17928-21apt-get08:01:103
2548999230,0cyclictest19208-21diskmemload10:16:583
2548999230,0cyclictest0-21swapper/312:25:153
2548999230,0cyclictest0-21swapper/311:57:183
2548999230,0cyclictest0-21swapper/311:01:523
2548999230,0cyclictest0-21swapper/310:54:433
2548999230,0cyclictest0-21swapper/309:49:433
2548999230,0cyclictest0-21swapper/307:36:083
25488992323,0cyclictest0-21swapper/212:06:192
25488992323,0cyclictest0-21swapper/211:48:542
25488992323,0cyclictest0-21swapper/210:05:242
25488992323,0cyclictest0-21swapper/209:40:372
25488992321,1cyclictest21148-21rm12:03:322
25488992320,2cyclictest15933-21ssh09:36:082
2548899230,2cyclictest26479-21ssh09:16:102
2548899230,23cyclictest0-21swapper/212:36:452
2548899230,23cyclictest0-21swapper/209:55:022
2548899230,23cyclictest0-21swapper/209:23:512
2548899230,1cyclictest0-21swapper/209:46:202
2548899230,0cyclictest0-21swapper/212:25:252
2548899230,0cyclictest0-21swapper/210:15:032
25487992323,0cyclictest0-21swapper/112:20:251
25487992323,0cyclictest0-21swapper/112:19:421
25487992323,0cyclictest0-21swapper/112:10:171
25487992323,0cyclictest0-21swapper/110:32:511
25487992323,0cyclictest0-21swapper/109:57:481
25487992322,1cyclictest19208-21diskmemload09:20:151
25487992322,1cyclictest0-21swapper/110:52:561
2548799230,2cyclictest653-21ssh10:47:211
2548799230,2cyclictest30671-21ssh11:42:551
2548799230,2cyclictest30671-21ssh11:42:551
2548799230,2cyclictest1728-21ssh10:20:381
2548799230,23cyclictest0-21swapper/110:57:221
2548799230,23cyclictest0-21swapper/110:18:211
2548799230,23cyclictest0-21swapper/110:11:061
2548799230,23cyclictest0-21swapper/109:16:171
2548799230,22cyclictest6655-21ssh09:26:241
2548799230,22cyclictest24240-21ssh09:42:281
2548799230,22cyclictest22562-21ssh12:05:061
2548799230,22cyclictest1742-21ssh09:51:141
2548799230,22cyclictest16956-21ssh12:00:061
2548799230,22cyclictest14256-21ssh11:00:041
2548799230,22cyclictest11137-21sh10:28:011
2548799230,0cyclictest19208-21diskmemload10:36:061
2548799230,0cyclictest19208-21diskmemload09:30:021
2548799230,0cyclictest0-21swapper/111:50:491
2548799230,0cyclictest0-21swapper/111:25:171
25482992323,0cyclictest19208-21diskmemload10:49:120
25482992323,0cyclictest19208-21diskmemload09:41:570
25482992323,0cyclictest0-21swapper/012:30:070
25482992323,0cyclictest0-21swapper/012:30:070
25482992323,0cyclictest0-21swapper/011:01:470
25482992322,1cyclictest62550irq/127-eno112:35:290
25482992322,1cyclictest62550irq/127-eno111:23:590
25482992322,1cyclictest25708-21kworker/u8:212:22:240
25482992322,1cyclictest19208-21diskmemload10:11:450
25482992321,1cyclictest0-21swapper/010:36:140
25482992321,1cyclictest0-21swapper/010:15:050
2548299230,2cyclictest3335-21ssh11:19:030
2548299230,2cyclictest24763-21cron08:15:000
2548299230,2cyclictest22398-21sh09:11:290
2548299230,23cyclictest28373-21sh10:43:370
2548299230,23cyclictest19208-21diskmemload10:50:320
2548299230,23cyclictest0-21swapper/012:05:440
2548299230,23cyclictest0-21swapper/011:40:350
2548299230,23cyclictest0-21swapper/011:40:350
2548299230,23cyclictest0-21swapper/010:31:440
2548299230,23cyclictest0-21swapper/009:45:010
2548299230,22cyclictest30470-21sh09:19:520
2548299230,22cyclictest0-21swapper/011:51:100
2548299230,22cyclictest0-21swapper/011:11:140
2548299230,0cyclictest0-21swapper/010:20:500
2548299230,0cyclictest0-21swapper/009:31:260
25489992222,0cyclictest19208-21diskmemload11:32:413
25489992222,0cyclictest0-21swapper/312:10:103
25489992220,1cyclictest30987-21rm09:20:203
2548999220,21cyclictest19208-21diskmemload11:35:213
2548999220,1cyclictest5008-21sh07:35:003
2548999220,1cyclictest0-21swapper/311:15:433
2548999220,1cyclictest0-21swapper/311:05:063
2548999220,1cyclictest0-21swapper/310:21:393
2548999220,1cyclictest0-21swapper/309:40:583
2548999220,0cyclictest19208-21diskmemload09:29:253
2548999220,0cyclictest0-21swapper/311:40:583
2548999220,0cyclictest0-21swapper/311:40:583
25488992222,0cyclictest19208-21diskmemload11:42:412
25488992222,0cyclictest19208-21diskmemload11:42:412
25488992222,0cyclictest0-21swapper/212:16:542
25488992222,0cyclictest0-21swapper/211:55:182
25488992222,0cyclictest0-21swapper/211:51:032
25488992222,0cyclictest0-21swapper/211:36:232
25488992222,0cyclictest0-21swapper/211:30:022
25488992222,0cyclictest0-21swapper/211:22:242
25488992222,0cyclictest0-21swapper/210:20:042
2548899220,2cyclictest6924-21ssh12:20:082
2548899220,21cyclictest14427-21ssh11:00:172
2548899220,21cyclictest13278-21sh10:30:312
2548899220,21cyclictest0-21swapper/211:15:032
2548899220,1cyclictest4152-21ssh09:25:182
2548899220,1cyclictest21387-21/usr/sbin/munin11:06:122
2548899220,1cyclictest15180-21perf07:55:012
2548899220,1cyclictest0-21swapper/212:30:082
2548899220,1cyclictest0-21swapper/212:30:082
2548899220,1cyclictest0-21swapper/210:40:362
2548899220,0cyclictest0-21swapper/211:29:452
2548899220,0cyclictest0-21swapper/209:50:122
25487992221,0cyclictest0-21swapper/111:06:091
25487992220,2cyclictest23004-21ssh09:12:141
25487992220,1cyclictest2607-21latency_hist07:30:001
2548799220,22cyclictest0-21swapper/111:56:451
2548799220,22cyclictest0-21swapper/111:20:361
2548799220,22cyclictest0-21swapper/111:19:221
2548799220,21cyclictest19208-21diskmemload10:02:101
2548799220,21cyclictest18853-21ssh12:30:261
2548799220,21cyclictest18853-21ssh12:30:261
2548799220,21cyclictest0-21swapper/109:45:401
2548799220,21cyclictest0-21swapper/109:36:061
2548799220,1cyclictest25747-21ssh11:10:311
2548799220,1cyclictest0-21swapper/111:45:541
25482992222,0cyclictest0-21swapper/009:58:350
25482992221,1cyclictest0-21swapper/012:12:000
25482992220,1cyclictest0-21swapper/012:25:250
25482992219,2cyclictest11402-21diskmemload11:55:120
2548299221,21cyclictest62550irq/127-eno112:16:030
2548299220,22cyclictest19208-21diskmemload10:55:550
2548299220,21cyclictest7124-21apt-get08:46:090
2548299220,21cyclictest14403-21kworker/u8:107:15:010
2548299220,21cyclictest0-21swapper/009:35:060
2548299220,1cyclictest0-21swapper/009:20:100
2548299220,0cyclictest0-21swapper/009:27:090
25489992121,0cyclictest0-21swapper/307:21:093
25489992120,1cyclictest25108-21apt-get08:16:103
25489992120,1cyclictest17173-21apt-get09:06:093
25489992118,2cyclictest11619-21runrttasks08:55:013
2548999210,21cyclictest0-21swapper/309:30:043
2548999210,20cyclictest0-21swapper/308:36:113
2548999210,20cyclictest0-21swapper/308:36:113
25488992121,0cyclictest0-21swapper/208:46:092
25488992120,1cyclictest0-21swapper/209:10:142
2548899210,20cyclictest10363-21awk07:45:002
2548899210,0cyclictest19208-21diskmemload12:10:472
2548899210,0cyclictest0-21swapper/208:41:102
2548799210,21cyclictest0-21swapper/110:06:121
2548799210,20cyclictest0-21swapper/109:01:111
2548799210,0cyclictest0-21swapper/108:01:101
2548799210,0cyclictest0-21swapper/107:31:091
25482992121,0cyclictest0-21swapper/008:56:100
25482992120,1cyclictest14782-21apt-get09:01:100
2548299210,21cyclictest0-21swapper/011:35:440
2548299210,20cyclictest0-21swapper/008:36:110
2548299210,20cyclictest0-21swapper/008:36:110
2548299210,0cyclictest0-21swapper/007:36:080
25489992020,0cyclictest0-21swapper/308:41:103
25489992017,2cyclictest2623-21ls07:30:003
25488992020,0cyclictest2317-21apt-get08:36:112
25488992020,0cyclictest2317-21apt-get08:36:112
25488992020,0cyclictest0-21swapper/208:51:102
25488992020,0cyclictest0-21swapper/207:56:092
2548899200,0cyclictest0-21swapper/209:01:102
2548899200,0cyclictest0-21swapper/208:55:022
25487992020,0cyclictest9520-21apt-get08:51:101
25487992020,0cyclictest22365-21perf08:10:001
25487992020,0cyclictest0-21swapper/108:41:101
25487992020,0cyclictest0-21swapper/107:11:081
25487992018,1cyclictest4361-21turbostat08:40:011
25487992018,1cyclictest4361-21turbostat08:40:011
25487992018,1cyclictest10395-21munin-run07:45:001
25482992020,0cyclictest4732-21apt-get08:41:100
25482992020,0cyclictest0-21swapper/008:16:100
25489991919,0cyclictest0-21swapper/309:01:113
25489991919,0cyclictest0-21swapper/307:46:093
25489991918,1cyclictest29931-21apt-get08:26:113
25489991917,1cyclictest27832-21awk07:15:013
25488991919,0cyclictest0-21swapper/208:16:102
25488991917,1cyclictest7928-21munin-run07:40:002
25488991917,1cyclictest32625-21apt07:25:002
25487991919,0cyclictest5019-21apt-get07:35:021
25487991919,0cyclictest27537-21apt-get08:21:091
25487991919,0cyclictest13115-21apt-get07:51:091
25487991918,1cyclictest10723-21apt-get07:46:091
25482991917,1cyclictest5030-21munin-plugin-st07:35:000
25482991917,1cyclictest29582-21idleruntime-cro08:25:000
25489991818,0cyclictest0-21swapper/308:11:073
25488991818,0cyclictest0-21swapper/208:21:102
25488991818,0cyclictest0-21swapper/207:50:012
25487991818,0cyclictest0-21swapper/108:31:101
25487991817,0cyclictest0-21swapper/108:55:021
25482991816,1cyclictest606-21dbus-daemon07:30:000
25482991815,2cyclictest19963-21cron08:05:010
25489991717,0cyclictest0-21swapper/308:21:093
25487991717,0cyclictest0-21swapper/107:56:091
25482991717,0cyclictest0-21swapper/007:56:080
25489991616,0cyclictest0-21swapper/308:31:103
25489991616,0cyclictest0-21swapper/307:55:013
25489991614,1cyclictest28807-21hddtemp_smartct07:16:113
25488991616,0cyclictest0-21swapper/208:06:082
25488991616,0cyclictest0-21swapper/207:26:102
2548899160,15cyclictest32756-21diskstats08:31:112
25487991616,0cyclictest0-21swapper/108:46:091
25487991614,1cyclictest17973-21irqstats09:06:131
25482991615,1cyclictest0-21swapper/007:16:090
25482991614,1cyclictest11639-21perf08:55:010
25487991515,0cyclictest21850irq/141-nvme0q208:20:001
25482991515,0cyclictest0-21swapper/007:51:090
25482991515,0cyclictest0-21swapper/007:21:090
25482991513,1cyclictest22041-21users08:06:180
25482991513,1cyclictest12758-21sh07:50:010
2548999140,14cyclictest0-21swapper/307:55:013
2548999140,0cyclictest0-21swapper/308:45:373
2548899140,14cyclictest0-21swapper/208:25:382
2548899140,14cyclictest0-21swapper/208:12:492
2548899140,14cyclictest0-21swapper/207:15:242
2548899140,14cyclictest0-21swapper/207:10:322
2548799140,14cyclictest0-21swapper/108:10:201
2548799140,14cyclictest0-21swapper/107:21:231
2548299140,14cyclictest0-21swapper/009:05:040
2548299140,14cyclictest0-21swapper/008:25:480
2548299140,14cyclictest0-21swapper/007:41:070
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional