You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-08 - 18:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Jun 08, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1561129769,9sleep00-21swapper/007:05:090
1761829265,9sleep20-21swapper/207:08:242
1715429164,9sleep30-21swapper/307:05:283
1764728263,15sleep10-21swapper/107:08:471
311452710,0sleep30-21swapper/312:17:233
161282540,1sleep30-21swapper/311:49:023
1783499270,1cyclictest0-21swapper/209:43:222
1783299271,6cyclictest24847-21/usr/sbin/munin10:55:201
1783499260,1cyclictest0-21swapper/211:13:372
1783499260,1cyclictest0-21swapper/209:18:272
1783299260,1cyclictest0-21swapper/112:35:181
1783499250,3cyclictest0-21swapper/212:32:022
1783499250,24cyclictest0-21swapper/210:27:472
1783499232,21cyclictest0-21swapper/211:25:582
1783499232,21cyclictest0-21swapper/211:25:582
1783499230,1cyclictest0-21swapper/209:57:052
1783499230,0cyclictest0-21swapper/212:01:582
1783299230,21cyclictest0-21swapper/111:35:271
17834992220,1cyclictest0-21swapper/212:28:302
1783499220,2cyclictest0-21swapper/212:20:382
1783499220,21cyclictest0-21swapper/211:04:172
1783299220,0cyclictest0-21swapper/110:34:411
1783499210,20cyclictest0-21swapper/211:20:442
17832992119,1cyclictest0-21swapper/109:45:201
1782999210,21cyclictest0-21swapper/009:34:410
1783899201,18cyclictest0-21swapper/311:14:203
17834992018,1cyclictest32-21ksoftirqd/210:20:252
1783499200,1cyclictest0-21swapper/211:15:122
1783499200,16cyclictest0-21swapper/210:16:532
1783299208,4cyclictest477-21rs:main1
17829992016,2cyclictest0-21swapper/009:14:450
1783499190,1cyclictest141rcu_preempt08:55:232
1783499190,1cyclictest0-21swapper/212:17:392
1783499190,1cyclictest0-21swapper/211:33:042
1783499190,1cyclictest0-21swapper/210:57:492
1783499190,1cyclictest0-21swapper/210:43:022
1783499190,18cyclictest0-21swapper/209:12:212
1783499190,17cyclictest0-21swapper/211:39:252
1783499190,16cyclictest0-21swapper/211:40:562
1783499190,16cyclictest0-21swapper/209:54:582
1783499190,0cyclictest0-21swapper/210:38:402
1783299190,3cyclictest0-21swapper/110:44:291
1783299190,1cyclictest0-21swapper/110:05:011
1783499180,17cyclictest0-21swapper/211:47:552
1783299181,16cyclictest26639-21cstates11:55:161
1783299180,0cyclictest0-21swapper/109:30:081
17829991817,1cyclictest0-21swapper/008:35:130
17834991715,1cyclictest55650irq/127-enp2s0-09:29:312
1783499170,13cyclictest9010-21sed10:45:262
1783499170,13cyclictest0-21swapper/210:51:142
1783299170,16cyclictest0-21swapper/111:06:131
1783299170,16cyclictest0-21swapper/111:03:581
17834991614,1cyclictest32-21ksoftirqd/209:23:042
1783499161,0cyclictest0-21swapper/209:35:412
1783499160,16cyclictest0-21swapper/210:30:452
1783499160,0cyclictest0-21swapper/209:34:082
17832991612,1cyclictest0-21swapper/111:23:251
1783299160,0cyclictest0-21swapper/112:01:321
1782999160,1cyclictest0-21swapper/012:03:090
17838991512,3cyclictest14775-21kworker/3:010:35:433
17838991510,3cyclictest24652-21ssh11:15:123
1783899150,15cyclictest0-21swapper/310:03:193
17832991513,1cyclictest0-21swapper/110:36:171
1783299151,0cyclictest3127-21kworker/1:211:32:521
1783299150,2cyclictest0-21swapper/112:20:511
1783299150,1cyclictest0-21swapper/109:53:081
1783299150,1cyclictest0-21swapper/109:20:061
17829991513,2cyclictest13-21ksoftirqd/010:54:490
209562140,3sleep232-21ksoftirqd/212:10:532
1783899148,4cyclictest12603-21/usr/sbin/munin09:50:243
1783899142,5cyclictest19655-21tr12:10:193
17838991410,3cyclictest5258-21sed09:45:163
17838991410,3cyclictest3454-21sed11:40:303
17838991410,2cyclictest18841-21wc08:20:003
1783499141,7cyclictest28539-21idleruntime-cro08:40:002
17834991410,3cyclictest30655-21grep08:40:262
1783499140,1cyclictest0-21swapper/211:59:352
17832991414,0cyclictest0-21swapper/108:20:191
17832991412,2cyclictest25-21ksoftirqd/112:34:071
1783299141,1cyclictest25880-21ssh09:57:311
17832991411,2cyclictest38150irq/131-snd_hda10:20:011
17832991411,2cyclictest14404-21ssh10:10:161
17832991410,3cyclictest9159-21awk11:45:011
1783299140,2cyclictest0-21swapper/111:29:021
1783299140,2cyclictest0-21swapper/111:29:021
1783299140,1cyclictest0-21swapper/110:51:181
1783299140,14cyclictest0-21swapper/109:15:011
1783299140,13cyclictest17350-21cron11:50:011
17829991413,1cyclictest0-21swapper/009:43:210
17829991413,1cyclictest0-21swapper/009:18:280
17829991412,2cyclictest13-21ksoftirqd/009:55:250
17829991411,2cyclictest24949-21latency_hist07:25:010
1782999140,3cyclictest0-21swapper/008:05:150
1782999140,2cyclictest0-21swapper/010:06:030
1782999140,13cyclictest0-21swapper/010:30:030
1783899139,3cyclictest20693-21grep12:30:213
1783899135,4cyclictest273-21systemd-udevd10:30:043
1783899135,4cyclictest225-21systemd-journal08:04:553
1783899133,7cyclictest21983-21grep09:55:183
1783899131,7cyclictest2968-21grep12:39:593
17838991312,1cyclictest39-21ksoftirqd/309:22:593
17838991310,2cyclictest9624-21wc11:05:223
17838991310,2cyclictest7601-21sed10:25:213
17838991310,2cyclictest2113-21munin-plugin-st07:45:003
17838991310,2cyclictest18834-21cut07:10:183
17838991310,2cyclictest13152-21grep09:30:203
17838991310,2cyclictest12307-21wc09:10:193
1783899130,8cyclictest5506-21wget10:05:143
1783899130,2cyclictest0-21swapper/311:20:563
1783899130,12cyclictest0-21swapper/310:10:433
1783499139,2cyclictest30057-21cut07:35:142
1783499135,6cyclictest0-21swapper/207:15:132
17834991310,2cyclictest627-21grep08:45:252
17834991310,2cyclictest5225-21grep09:45:162
17834991310,2cyclictest31029-21grep10:00:192
17834991310,2cyclictest2922-21cron12:40:002
17834991310,2cyclictest14882-21sort08:10:162
1783299139,3cyclictest30173-21tr08:40:211
1783299139,3cyclictest28683-21cat08:40:001
1783299138,4cyclictest28122-21ssh10:18:391
1783299138,4cyclictest13639-21sh10:49:011
1783299138,4cyclictest10429-21taskset09:07:421
1783299135,4cyclictest225-21systemd-journal08:05:001
17832991310,2cyclictest5565-21grep09:25:261
17832991310,2cyclictest30647-21grep07:35:161
17832991310,2cyclictest1247-21cat08:50:001
1783299130,0cyclictest0-21swapper/112:12:591
1782999139,3cyclictest6312-21cat07:50:250
17829991312,1cyclictest0-21swapper/011:13:370
17829991311,1cyclictest3587-21apt-get09:25:120
17829991310,2cyclictest28771-21uname09:20:180
17829991310,2cyclictest25885-21cron08:35:000
17829991310,2cyclictest22671-21wc08:25:210
1782999130,10cyclictest0-21swapper/011:50:080
1783899129,2cyclictest7855-21grep07:55:183
1783899129,2cyclictest698-21grep11:00:193
1783899129,2cyclictest5496-21grep07:50:163
1783899129,2cyclictest4411-21grep08:55:173
1783899129,2cyclictest3388-21sed12:00:223
1783899129,2cyclictest23528-21awk08:30:003
1783899129,2cyclictest16475-21wc08:15:003
1783899129,2cyclictest1464-21ls12:00:013
1783899129,2cyclictest1361-21taskset10:40:493
1783899129,2cyclictest12768-21sed12:25:263
1783899129,2cyclictest10330-21ls12:25:003
1783899128,3cyclictest31537-21latency_hist11:00:003
1783899125,3cyclictest26381-21apt-get08:35:143
1783899124,4cyclictest225-21systemd-journal09:29:593
1783899122,7cyclictest0-21swapper/312:05:183
1783899122,5cyclictest19022-21grep11:30:263
1783899121,6cyclictest10634-21diskmemload11:35:123
17838991211,1cyclictest15361-21kworker/3:209:15:433
1783899120,12cyclictest17750-21sh10:50:453
1783899120,11cyclictest16656-21grep11:30:003
1783899120,11cyclictest16656-21grep11:30:003
1783499129,2cyclictest23617-21sed07:20:172
1783499129,2cyclictest22552-21tr08:25:202
1783499129,2cyclictest18919-21wc08:20:002
1783499128,3cyclictest2095-21cron07:45:002
1783499128,2cyclictest25357-21cut07:25:152
1783499120,5cyclictest225-21systemd-journal08:50:132
1783499120,5cyclictest225-21systemd-journal08:50:132
1783499120,2cyclictest0-21swapper/210:12:252
1783299129,3cyclictest28024-21apache212:06:151
1783299129,2cyclictest3469-21grep07:45:191
1783299129,2cyclictest28815-21grep09:40:131
1783299129,2cyclictest21322-21ssh09:15:261
1783299129,2cyclictest19090-21grep11:50:191
1783299129,2cyclictest15830-21wc08:10:251
1783299129,2cyclictest12085-21awk12:25:201
1783299128,3cyclictest23109-21head08:25:261
1783299127,4cyclictest31199-21tr10:00:221
1782999129,2cyclictest5918-21ssh10:25:010
1782999129,2cyclictest3469-21cron08:55:000
1782999129,2cyclictest3469-21cron08:55:000
1782999129,2cyclictest14027-21cut10:10:140
1782999129,2cyclictest10539-21grep12:05:150
1782999129,2cyclictest10081-21cat09:05:250
17829991211,1cyclictest0-21swapper/011:32:570
17829991211,1cyclictest0-21swapper/011:18:300
17829991210,2cyclictest13-21ksoftirqd/012:32:380
17829991210,1cyclictest0-21swapper/010:40:270
1782999120,3cyclictest0-21swapper/010:18:070
1783899119,1cyclictest81432sleep310:45:183
1783899118,2cyclictest901-21ssh10:21:133
1783899118,2cyclictest7738-21grep09:00:243
1783899118,2cyclictest32232-21sort08:45:173
1783899118,2cyclictest3171-21taskset08:51:113
1783899118,2cyclictest3171-21taskset08:51:103
1783899118,2cyclictest30700-21sort07:35:173
1783899118,2cyclictest30134-21python09:40:233
1783899118,2cyclictest29219-21cat07:30:263
1783899118,2cyclictest25388-21ls08:30:253
1783899118,2cyclictest23626-21tail07:20:173
1783899118,2cyclictest22382-21expr10:15:143
1783899118,2cyclictest20985-21ssh11:51:193
1783899118,2cyclictest20799-21sed09:35:163
1783899118,2cyclictest19282-21expr08:20:143
1783499119,1cyclictest9312-21wc08:00:002
1783499118,2cyclictest8792-21tr09:05:152
1783499118,2cyclictest6134-21grep10:05:172
1783499118,2cyclictest4881-21cut07:50:132
1783499118,2cyclictest24768-21ssh11:54:172
1783499118,2cyclictest24497-21tail08:30:162
1783499118,2cyclictest20151-21chrt07:14:552
1783499118,2cyclictest13527-21tail08:05:282
1783499118,2cyclictest13107-21sh12:06:082
1783299118,2cyclictest8349-21grep07:55:231
1783299118,2cyclictest8075-21grep10:25:261
1783299118,2cyclictest26036-21basename07:25:191
1783299117,1cyclictest0-21swapper/112:15:131
1783299116,4cyclictest8153-21taskset09:03:571
1783299110,6cyclictest21860-21grep09:35:271
1783299110,1cyclictest0-21swapper/111:12:091
1782999119,2cyclictest0-21swapper/012:37:490
1782999119,1cyclictest71152sleep009:46:010
1782999119,1cyclictest31114-21ntpq10:00:210
1782999118,2cyclictest8743-21grep07:55:270
1782999118,2cyclictest4242-21cat12:20:270
1782999118,2cyclictest3689-21ssh11:40:400
1782999118,2cyclictest29363-21expr08:40:150
1782999118,2cyclictest25925-21grep07:25:170
1782999118,2cyclictest23832-21basename10:55:150
1782999118,2cyclictest2052-21basename12:20:000
1782999118,2cyclictest10447-21head11:45:180
1782999118,2cyclictest0-21swapper/010:25:000
1782999117,3cyclictest26893-21sh11:55:180
1782999116,4cyclictest4495-21idleruntime-cro07:50:000
17829991110,1cyclictest0-21swapper/010:37:190
1782999110,1cyclictest0-21swapper/011:35:360
1783899108,1cyclictest9938-21cat09:05:253
1783899108,1cyclictest25912-21tune2fs07:25:183
1783899107,2cyclictest4128-21chrt07:45:383
1783899107,2cyclictest29012-21acpi08:40:123
1783899107,2cyclictest22382-21taskset07:17:593
1783899107,2cyclictest13118-21grep08:05:233
1783499108,1cyclictest11001-21sed08:00:252
1783499107,2cyclictest8147-21ssh11:05:132
1783499107,2cyclictest6379-21cut09:00:132
1783499107,2cyclictest4088-21head07:45:262
1783499107,2cyclictest27866-21expr07:30:142
1783499107,2cyclictest19801-21taskset08:20:182
1783299108,1cyclictest6630-21chrt07:52:181
1783299108,1cyclictest581-21taskset07:40:171
1783299108,1cyclictest4943-21cut08:55:221
1783299108,1cyclictest27780-21grep07:30:141
1783299108,1cyclictest23792-21chrt08:30:121
1783299108,1cyclictest19223-21ntpq07:10:221
1783299108,1cyclictest1335-21taskset08:50:111
1783299108,1cyclictest1335-21taskset08:50:111
1783299108,1cyclictest12668-21expr08:05:181
1783299107,2cyclictest22963-21grep07:20:141
1783299107,2cyclictest21603-21sh07:15:211
1782999109,1cyclictest0-21swapper/011:20:440
1782999108,2cyclictest404-21expr07:40:150
1782999108,2cyclictest22320-21chrt07:17:110
1782999108,2cyclictest19661-21ls07:10:260
1782999108,1cyclictest9534-21hostname08:00:110
1782999108,1cyclictest5619-21taskset08:57:110
1782999108,1cyclictest41862sleep011:02:180
1782999108,1cyclictest30304-21grep07:35:150
1782999108,1cyclictest29324-21sed07:30:260
1782999108,1cyclictest19752-21tail08:20:170
1782999108,1cyclictest12801-21ssh12:25:270
1782999108,1cyclictest10863-21sh11:25:260
1782999108,1cyclictest10863-21sh11:25:250
1782999107,2cyclictest19471-21sort12:10:180
1782999107,2cyclictest18414-21taskset08:15:510
1782999101,6cyclictest1142-21runrttasks09:37:020
1782999100,9cyclictest14263-21apt-get08:10:120
1782999100,9cyclictest0-21swapper/011:07:560
1782999100,1cyclictest0-21swapper/009:53:210
178329997,1cyclictest17483-21cat08:15:171
178299998,1cyclictest7311-21cpu10:45:140
178299998,1cyclictest32339-21uname08:45:180
178299997,1cyclictest61742sleep009:00:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional