You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-25 - 16:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Sat Jul 25, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2890129568,8sleep00-21swapper/007:09:030
2871829583,8sleep20-21swapper/207:06:402
2883129364,10sleep30-21swapper/307:08:103
2869028972,13sleep10-21swapper/107:06:191
29065992724,2cyclictest13-21ksoftirqd/010:15:100
2907399226,11cyclictest12393-21apt-get08:50:163
2907399221,0cyclictest0-21swapper/309:43:513
2907399220,1cyclictest0-21swapper/309:59:093
2907399211,3cyclictest0-21swapper/309:23:553
2907399210,0cyclictest0-21swapper/310:49:323
2907099210,2cyclictest6742-21/usr/sbin/munin09:20:202
29073992020,0cyclictest0-21swapper/311:11:533
29070991916,3cyclictest32-21ksoftirqd/211:03:112
2906799191,1cyclictest6606-21aten2_r7power_e11:35:151
2906799190,1cyclictest0-21swapper/110:14:441
2906799190,19cyclictest0-21swapper/111:29:461
2906799190,18cyclictest0-21swapper/111:22:151
2907099188,3cyclictest21432-21perf11:25:002
2907099180,18cyclictest0-21swapper/211:46:572
2906799180,1cyclictest0-21swapper/112:14:591
2906799180,1cyclictest0-21swapper/112:14:591
2906799180,18cyclictest0-21swapper/112:07:151
2906799180,17cyclictest0-21swapper/108:15:191
2906599184,6cyclictest24480-21perf09:50:000
2906599180,18cyclictest0-21swapper/011:45:320
29073991716,1cyclictest0-21swapper/309:52:203
2907399170,17cyclictest0-21swapper/312:04:463
2907399170,0cyclictest0-21swapper/311:42:453
29067991716,1cyclictest0-21swapper/111:34:061
2906799171,16cyclictest17406-21ssh11:02:391
2906799170,17cyclictest0-21swapper/109:20:321
29073991615,1cyclictest0-21swapper/310:10:163
2907099163,13cyclictest0-21swapper/212:31:002
2907099160,16cyclictest0-21swapper/210:35:312
2907099160,16cyclictest0-21swapper/210:35:312
2906799162,14cyclictest23346-21ssh11:05:461
29067991615,1cyclictest25-21ksoftirqd/109:33:331
2906799160,1cyclictest0-21swapper/112:35:151
2906599165,2cyclictest0-21swapper/009:35:130
29065991614,2cyclictest13-21ksoftirqd/012:13:540
29065991614,2cyclictest13-21ksoftirqd/012:13:530
2906599160,1cyclictest0-21swapper/012:04:570
29073991513,2cyclictest39-21ksoftirqd/311:17:163
29073991511,3cyclictest9239-21wc11:55:263
2907399150,8cyclictest0-21swapper/311:50:163
2907399150,2cyclictest0-21swapper/309:27:583
2907399150,2cyclictest0-21swapper/309:10:433
2907399150,15cyclictest0-21swapper/309:16:053
29070991514,1cyclictest0-21swapper/210:25:042
29070991511,3cyclictest17955-21uniq12:20:182
29067991514,1cyclictest0-21swapper/110:31:011
2906599152,9cyclictest0-21swapper/007:14:350
29065991515,0cyclictest0-21swapper/012:36:150
29065991510,3cyclictest19782-21grep09:05:150
2906599150,7cyclictest0-21swapper/009:15:160
2906599150,2cyclictest0-21swapper/010:42:510
2906599150,13cyclictest0-21swapper/011:35:240
29073991411,2cyclictest7758-21grep11:35:213
29073991410,3cyclictest3092-21cat10:35:133
29073991410,3cyclictest3092-21cat10:35:133
29073991410,2cyclictest13874-21ssh11:00:213
2907399140,1cyclictest0-21swapper/311:05:403
2907399140,0cyclictest0-21swapper/311:32:323
2907099148,5cyclictest18932-21needreboot10:05:222
2907099141,12cyclictest0-21swapper/211:30:162
29067991413,1cyclictest0-21swapper/109:40:351
29067991410,3cyclictest31326-21grep11:10:291
29067991410,3cyclictest20873-21cat08:00:001
29067991410,3cyclictest11262-21grep10:00:291
2906799140,13cyclictest0-21swapper/111:47:331
29065991411,2cyclictest30362-21wc09:15:000
29065991411,2cyclictest26320-21cut12:25:180
29065991410,3cyclictest2944-21grep08:30:160
2906599140,8cyclictest0-21swapper/012:30:150
311492130,0chrt0-21swapper/010:32:020
2907399139,3cyclictest25841-21ssh12:05:253
2907399139,3cyclictest17841-21wc09:45:213
2907399139,2cyclictest31402-21apt07:15:003
2907399139,2cyclictest16105-21wc07:50:003
2907399138,4cyclictest2293-21fschecks_time12:30:183
2907399132,3cyclictest225-21systemd-journal10:55:003
29073991312,1cyclictest0-21swapper/312:17:053
29073991312,1cyclictest0-21swapper/308:26:073
29073991310,2cyclictest7274-21sort07:30:173
29073991310,2cyclictest30515-21basename08:20:143
29073991310,2cyclictest29762-21cron11:30:003
29073991310,2cyclictest28186-21grep08:15:153
29073991310,2cyclictest25956-21ls12:25:163
29073991310,2cyclictest20970-21grep09:05:233
29073991310,2cyclictest16373-21grep08:55:263
29073991310,2cyclictest11839-21cut10:40:163
29073991310,2cyclictest10309-21wc07:35:223
2907099139,3cyclictest6142-21grep10:55:262
2907099138,3cyclictest23727-21ssh09:30:142
2907099137,5cyclictest2516-21apache210:44:052
2907099131,7cyclictest15029-21cut11:40:162
2907099131,11cyclictest17406-21awk12:00:212
29070991310,2cyclictest8271-21grep11:55:172
29070991310,2cyclictest31903-21grep11:50:172
29070991310,2cyclictest30351-21cat07:10:192
29070991310,2cyclictest28860-21grep08:15:182
29070991310,2cyclictest25587-21cat08:10:002
29070991310,2cyclictest22829-21grep11:05:262
29070991310,2cyclictest13555-21turbostat07:45:002
2906799139,3cyclictest19472-21cron09:05:001
2906799139,3cyclictest17730-21cat09:45:201
2906799139,3cyclictest17723-21timerwakeupswit07:50:261
2906799133,9cyclictest38150irq/131-snd_hda11:52:151
29067991313,0cyclictest0-21swapper/110:48:091
29067991311,2cyclictest5206-21kworker/1:009:10:421
29067991310,2cyclictest7410-21grep07:30:171
29067991310,2cyclictest5924-21grep07:25:271
29067991310,2cyclictest4332-21wc10:35:221
29067991310,2cyclictest4332-21wc10:35:221
29067991310,2cyclictest30712-21wc07:10:261
2906599139,3cyclictest20679-21uniq10:45:180
2906599138,3cyclictest10116-21grep08:45:140
29065991310,2cyclictest25332-21grep09:50:170
29065991310,2cyclictest17296-21uname07:50:200
29065991310,2cyclictest1554-21wc07:20:010
2906599130,12cyclictest12070-21cron11:00:010
2907399129,2cyclictest8233-21dpkg12:15:003
2907399129,2cyclictest8233-21dpkg12:15:003
2907399129,2cyclictest5021-21tail07:25:183
2907399129,2cyclictest2737-21cut10:15:163
2907399129,2cyclictest24574-21head09:30:193
2907399129,2cyclictest19971-21chrt12:21:003
2907399129,2cyclictest18560-21cut10:05:193
2907399129,2cyclictest16291-21grep07:50:143
2907399129,2cyclictest12126-21ssh10:20:263
2907399129,2cyclictest11846-21grep12:35:293
2907399122,3cyclictest0-21swapper/309:35:143
2907099129,2cyclictest5028-21head07:25:182
2907099129,2cyclictest32156-21sed08:20:282
2907099129,2cyclictest30807-21chrt10:12:362
2907099129,2cyclictest2660-21grep07:20:182
2907099129,2cyclictest2597-21cron08:30:002
2907099129,2cyclictest25625-21tr12:05:232
2907099129,2cyclictest23687-21ntpq09:10:222
2907099129,2cyclictest21909-21apache_volume11:25:112
2907099129,2cyclictest1397-21tr07:20:002
2907099124,7cyclictest31410-21/usr/sbin/munin09:15:162
2907099122,9cyclictest0-21swapper/209:58:292
2907099120,8cyclictest0-21swapper/208:55:132
2907099120,11cyclictest0-21swapper/210:45:492
2906799129,2cyclictest7221-21chrt10:17:561
2906799129,2cyclictest6672-21head11:15:231
2906799129,2cyclictest3423-21grep08:30:171
2906799129,2cyclictest31837-21grep08:20:241
2906799129,2cyclictest27350-21sed08:10:281
2906799129,2cyclictest25266-21sort12:25:011
2906799129,2cyclictest1369-21apt07:20:001
2906799129,2cyclictest1068-21tr09:55:161
2906799128,3cyclictest25870-21ssh10:29:271
2906799127,4cyclictest20633-21ssh11:43:241
2906799127,4cyclictest18895-21ssh12:01:121
2906799127,4cyclictest14180-21expr07:45:161
2906799127,4cyclictest11704-21ssh11:57:071
2906799125,5cyclictest537-21snmpd09:35:231
2906799122,9cyclictest23580-21sh10:08:291
2906799121,7cyclictest225-21systemd-journal12:20:001
29067991210,2cyclictest0-21swapper/110:20:261
2906599129,2cyclictest8819-21head11:55:200
2906599129,2cyclictest8341-21hostname09:40:140
2906599129,2cyclictest6419-21awk07:30:010
2906599129,2cyclictest3140-21sed07:20:220
2906599129,2cyclictest30776-21grep08:20:160
2906599129,2cyclictest29296-21cut08:15:230
2906599129,2cyclictest23579-21tr08:05:180
2906599129,2cyclictest23058-21chrt12:23:260
2906599129,2cyclictest21079-21cut11:05:130
2906599129,2cyclictest19594-21tail10:25:180
2906599129,2cyclictest17959-21tr10:05:170
2906599129,2cyclictest16862-21cron09:00:000
2906599129,2cyclictest15021-21cut11:40:160
2906599129,2cyclictest12402-21sed10:20:270
2906599128,3cyclictest10653-21tr10:00:230
2906599120,12cyclictest0-21swapper/009:55:370
2906599120,0cyclictest0-21swapper/011:01:450
2907399118,2cyclictest7750-21grep08:40:153
2907399118,2cyclictest7103-21taskset08:36:533
2907399118,2cyclictest5824-21ssh10:55:243
2907399118,2cyclictest32224-21sed07:15:143
2907399118,2cyclictest29875-21ssh10:31:023
2907399118,2cyclictest2661-21tail07:20:173
2907399118,2cyclictest24710-21ssh11:45:233
2907399118,2cyclictest20037-21ntpq10:25:213
2907399118,2cyclictest19267-21grep07:55:183
2907399118,2cyclictest16208-21ssh11:20:543
2907399118,2cyclictest10072-21cut08:45:133
2907399118,2cyclictest10006-21ssh10:00:183
2907399110,10cyclictest19306-21taskset09:02:543
2907099118,2cyclictest7132-21rm09:39:312
2907099118,2cyclictest6518-21strings11:15:212
2907099118,2cyclictest6357-21ssh10:17:152
2907099118,2cyclictest32464-21rm10:33:032
2907099118,2cyclictest30124-21tr11:10:202
2907099118,2cyclictest26695-21cat09:50:282
2907099118,2cyclictest26529-21basename08:10:202
2907099118,2cyclictest25751-21grep12:25:162
2907099118,2cyclictest19690-21ntpq07:55:222
2907099118,2cyclictest17965-21taskset07:50:532
2907099118,2cyclictest17342-21sort09:45:182
2907099118,2cyclictest15085-21ssh10:03:242
2907099118,2cyclictest14468-21ssh09:43:442
2907099118,2cyclictest13260-21ssh11:39:102
2907099118,2cyclictest12426-21tr08:50:162
2907099118,2cyclictest11330-21ls12:35:242
2907099118,2cyclictest1115-21sed12:10:192
2907099118,2cyclictest1115-21sed12:10:192
2907099118,2cyclictest10900-21cat08:45:182
2907099110,10cyclictest30516-21sh10:50:392
2907099110,10cyclictest19464-21ssh09:27:062
2907099110,10cyclictest10546-21aten2_r7power_p10:20:152
2906799119,1cyclictest4020-21chrt12:30:481
2906799118,2cyclictest9118-21rm10:57:431
2906799118,2cyclictest28018-21ssh10:50:041
2906799118,2cyclictest26321-21sh12:25:181
2906799118,2cyclictest24150-21chrt08:05:211
2906799118,2cyclictest20979-21cut08:00:131
2906799118,2cyclictest19105-21ssh09:26:501
2906799118,2cyclictest13369-21timerwakeupswit10:40:281
2906799118,2cyclictest13160-21sed07:40:271
29067991110,1cyclictest25040-21ssh09:50:161
2906599118,2cyclictest8585-21ssh11:16:280
2906599118,2cyclictest8351-21tail08:40:180
2906599118,2cyclictest7919-21cat09:20:200
2906599118,2cyclictest6960-21head08:35:270
2906599118,2cyclictest5210-21ssh10:35:300
2906599118,2cyclictest5210-21ssh10:35:300
2906599118,2cyclictest32286-21cut11:50:180
2906599118,2cyclictest31680-21sh10:13:180
2906599118,2cyclictest31319-21grep11:30:190
2906599118,2cyclictest30491-21ssh11:10:220
2906599118,2cyclictest29776-21awk11:30:000
2906599118,2cyclictest29163-21tail10:50:210
2906599118,2cyclictest24966-21cut09:30:210
2906599118,2cyclictest17577-21chrt11:21:590
2906599118,2cyclictest16010-21ssh09:25:170
2906599118,2cyclictest13997-21sed08:50:290
2906599117,3cyclictest25148-21grep12:05:200
2906599112,6cyclictest18136-21gltestperf09:00:180
84452100,0sleep10-21swapper/108:40:201
2907399108,1cyclictest132292sleep307:41:033
2907399107,2cyclictest5089-21cat08:35:003
2907399107,2cyclictest22887-21taskset08:02:533
2907399100,9cyclictest23363-21cut08:05:173
2907099108,1cyclictest9274-21expr12:15:152
2907099108,1cyclictest9153-21wget07:35:142
2907099108,1cyclictest7855-21cpuspeed_turbos08:40:162
2907099108,1cyclictest6636-21taskset08:35:242
2907099108,1cyclictest17608-21grep09:00:162
2907099107,2cyclictest7148-21cat07:30:152
2907099107,2cyclictest4722-21chrt08:31:462
2907099107,2cyclictest21551-21taskset09:06:142
2907099107,2cyclictest15442-21grep07:45:282
2907099105,4cyclictest23076-21ls08:05:002
2906799108,1cyclictest972-21df_abs08:25:161
2906799108,1cyclictest9412-21expr07:35:161
2906799108,1cyclictest5645-21expr08:35:161
2906799108,1cyclictest3625-21sed07:20:271
2906799108,1cyclictest30813-21cut09:15:141
2906799108,1cyclictest21368-21grep09:05:271
2906799108,1cyclictest16558-21seq08:56:081
2906799108,1cyclictest14656-21cat08:55:001
2906799108,1cyclictest10833-21basename08:45:171
2906599108,1cyclictest18706-21grep07:55:160
2906599108,1cyclictest11757-21wc07:40:160
2906599107,2cyclictest9028-21wget12:15:140
2906599107,2cyclictest6999-21expr07:30:150
2906599107,2cyclictest21057-21grep08:00:140
2906599107,2cyclictest14535-21grep07:45:180
2906599106,3cyclictest10618-21wc07:35:270
2906599105,3cyclictest25974-21grep08:10:170
290739997,1cyclictest25660-21date08:10:003
290659986,1cyclictest1330-21taskset08:25:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional