You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-19 - 19:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Sun Jul 19, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
573728574,6sleep00-21swapper/007:09:110
570528460,8sleep30-21swapper/307:08:463
559028269,9sleep20-21swapper/207:07:152
574827759,14sleep10-21swapper/107:09:201
160562720,0sleep30-21swapper/309:59:023
304292640,0sleep30-21swapper/312:22:363
589499300,29cyclictest0-21swapper/210:55:312
5887992726,1cyclictest0-21swapper/012:11:540
589499250,25cyclictest0-21swapper/211:48:302
589299250,3cyclictest0-21swapper/112:18:391
589799244,19cyclictest17882-21ssh11:17:093
589299242,19cyclictest0-21swapper/108:55:021
589799231,21cyclictest24527-21ssh11:20:473
589499230,23cyclictest0-21swapper/211:20:322
589299220,21cyclictest0-21swapper/112:14:231
589299220,1cyclictest0-21swapper/111:20:181
589499210,20cyclictest0-21swapper/212:10:022
589299210,20cyclictest12249-21cpu08:30:151
5897992018,1cyclictest0-21swapper/310:30:053
589799200,19cyclictest0-21swapper/311:50:153
589499200,19cyclictest0-21swapper/212:35:092
589499200,19cyclictest0-21swapper/211:51:542
5892992019,1cyclictest0-21swapper/109:36:531
5887992018,2cyclictest0-21swapper/009:26:290
589799190,19cyclictest0-21swapper/309:44:303
589799190,18cyclictest28343-21kworker/3:111:25:053
589499190,19cyclictest0-21swapper/209:38:302
589299199,6cyclictest0-21swapper/110:36:021
589299191,17cyclictest17715-21sh11:55:221
589299190,2cyclictest0-21swapper/112:30:141
589799182,11cyclictest0-21swapper/311:10:133
589499182,1cyclictest141rcu_preempt10:18:362
589499180,18cyclictest0-21swapper/209:45:332
589499180,17cyclictest0-21swapper/210:45:402
589499180,16cyclictest0-21swapper/211:36:032
589499180,16cyclictest0-21swapper/211:36:022
5892991812,5cyclictest23653-21sh09:24:561
589299180,1cyclictest0-21swapper/111:29:461
5887991817,1cyclictest0-21swapper/010:55:310
5887991816,1cyclictest0-21swapper/009:19:420
589799172,1cyclictest0-21swapper/309:22:503
589799170,13cyclictest0-21swapper/311:05:133
589499173,13cyclictest0-21swapper/209:30:162
5894991713,3cyclictest0-21swapper/212:01:342
589499171,13cyclictest0-21swapper/210:01:032
589499170,2cyclictest0-21swapper/210:28:302
589499170,17cyclictest0-21swapper/210:12:502
5892991712,4cyclictest0-21swapper/111:01:061
5887991717,0cyclictest0-21swapper/010:06:170
589799162,13cyclictest0-21swapper/312:25:303
5897991613,2cyclictest0-21swapper/310:15:123
589799160,1cyclictest0-21swapper/310:41:243
589799160,1cyclictest0-21swapper/309:17:563
5894991615,1cyclictest291irq_work/212:32:102
589499160,1cyclictest0-21swapper/210:38:372
589499160,16cyclictest0-21swapper/211:14:452
589499160,16cyclictest0-21swapper/209:22:262
589499160,15cyclictest0-21swapper/212:25:152
589299166,8cyclictest38150irq/131-snd_hda11:10:141
5892991616,0cyclictest0-21swapper/110:53:431
5892991614,1cyclictest6340-21sed08:15:241
589299161,14cyclictest0-21swapper/111:05:251
589299160,16cyclictest0-21swapper/111:35:231
589299160,16cyclictest0-21swapper/111:35:231
589799152,13cyclictest0-21swapper/311:02:123
5897991510,3cyclictest10822-21grep07:20:003
589799150,1cyclictest0-21swapper/310:29:463
589799150,0cyclictest0-21swapper/312:03:053
5894991510,5cyclictest449-21irqbalance09:14:022
589499150,2cyclictest225-21systemd-journal09:44:572
589499150,12cyclictest0-21swapper/209:30:002
589299154,4cyclictest225-21systemd-journal08:40:011
589299152,12cyclictest6214-21ssh10:31:081
5892991513,1cyclictest0-21swapper/111:30:171
5892991512,2cyclictest25531-21ntp_kernel_pll_09:25:191
5892991511,2cyclictest9205-21cron08:25:001
589299150,1cyclictest0-21swapper/110:43:231
588799158,6cyclictest477-21rs:main0
5887991514,1cyclictest0-21swapper/010:14:260
589799143,7cyclictest18935-21ssh12:35:153
5897991410,3cyclictest26226-21sed09:25:263
5897991410,1cyclictest16227-21ssh12:14:353
589799140,1cyclictest0-21swapper/308:00:133
589499147,5cyclictest5882-21cyclictest11:30:002
5894991412,1cyclictest32-21ksoftirqd/210:34:322
589499141,1cyclictest141rcu_preempt12:24:292
5894991411,2cyclictest15372-21grep11:15:252
589499140,14cyclictest0-21swapper/209:59:352
589299141,7cyclictest0-21swapper/110:10:121
589299141,2cyclictest11476-21hddtemp_smartct10:15:161
589299141,11cyclictest31048-21diskmemload10:24:501
5892991410,3cyclictest27772-21tr07:55:151
588799147,6cyclictest30496-21apt-get11:25:130
5887991413,1cyclictest0-21swapper/011:48:290
588799140,4cyclictest0-21swapper/011:34:180
589799139,3cyclictest19203-21grep12:15:263
5897991311,1cyclictest0-21swapper/312:30:063
5897991310,2cyclictest4021-21grep08:10:273
5897991310,2cyclictest29683-21awk08:00:003
5897991310,2cyclictest16143-21grep08:35:263
5897991310,2cyclictest1249-21grep12:05:173
5897991310,2cyclictest1112-21grep11:45:273
589799130,2cyclictest0-21swapper/311:35:413
589799130,2cyclictest0-21swapper/311:35:413
589799130,13cyclictest0-21swapper/309:37:563
5894991310,2cyclictest6759-21tr11:30:152
5894991310,2cyclictest30208-21tr08:00:152
5894991310,2cyclictest26872-21cut10:05:162
5894991310,2cyclictest21521-21grep07:40:202
589499130,12cyclictest8403-21grep12:10:002
589299139,3cyclictest21550-21run-parts11:20:001
589299139,3cyclictest15089-21chrt09:58:161
589299139,2cyclictest411-21tr11:45:211
589299138,3cyclictest9713-21grep08:25:171
589299132,1cyclictest31048-21diskmemload09:50:421
5892991311,2cyclictest38150irq/131-snd_hda12:38:311
5892991310,2cyclictest6835-21grep07:10:171
5892991310,2cyclictest26382-21ls07:50:221
5892991310,2cyclictest18652-21grep10:00:171
589299130,11cyclictest23513-21tail07:45:181
588799131,6cyclictest32057-21munin-run08:05:000
5887991312,1cyclictest0-21swapper/011:20:320
5887991310,2cyclictest8010-21chrt12:09:430
5887991310,2cyclictest13039-21cat07:25:000
589799129,2cyclictest9752-21cut08:25:173
589799129,2cyclictest6841-21grep07:10:173
589799129,2cyclictest3476-21cut10:10:223
589799129,2cyclictest28666-21sh09:04:593
589799129,2cyclictest27694-21ssh10:05:233
589799129,2cyclictest23920-21sed08:55:003
589799129,2cyclictest23282-21df07:45:173
589799129,2cyclictest17115-21grep08:40:153
589799129,2cyclictest16362-21head07:30:193
589799129,2cyclictest13547-21grep10:35:253
589799129,2cyclictest11142sleep309:10:263
589799128,3cyclictest26933-21wc10:25:013
589799128,2cyclictest25415-21cut07:50:153
589799120,11cyclictest12343-21grep08:30:153
589499129,2cyclictest867-21wc08:05:202
589499129,2cyclictest7834-21tr09:15:152
589499129,2cyclictest5448-21sed08:15:152
589499129,2cyclictest30575-21grep11:05:202
589499129,2cyclictest27236-21cut09:00:172
589499129,2cyclictest20228-21grep10:20:202
589499129,2cyclictest13294-21tr08:30:222
589499128,3cyclictest29202-21tr09:05:152
589499128,3cyclictest17122-21tr08:40:152
589499128,3cyclictest14970-21grep07:25:262
589499120,0cyclictest0-21swapper/210:41:432
589299129,2cyclictest7626-21ssh09:15:141
589299129,2cyclictest586-21cron09:50:001
589299129,2cyclictest2543-21grep08:10:151
589299129,2cyclictest24730-21wc09:45:001
589299129,2cyclictest2432-21grep12:05:261
589299129,2cyclictest2024-21cat08:10:001
589299129,2cyclictest13515-21grep07:25:141
589299128,2cyclictest27527-21sed10:05:221
589299126,3cyclictest28299-21kworker/1:109:34:331
589299122,9cyclictest7053-21perf09:15:001
589299122,9cyclictest2115-21kworker/1:012:23:151
589299120,1cyclictest0-21swapper/107:23:431
588799129,2cyclictest32533-21tr08:05:160
588799129,2cyclictest28139-21grep07:55:180
588799129,2cyclictest25802-21tr12:00:210
588799129,2cyclictest24817-21grep08:55:180
588799129,2cyclictest21344-21uname07:40:190
588799129,2cyclictest19551-21cat10:00:260
588799129,2cyclictest18730-21grep07:35:170
588799129,2cyclictest16428-21grep07:30:190
5887991210,1cyclictest24895-21cron07:49:590
5887991210,1cyclictest0-21swapper/008:35:130
589799118,2cyclictest7653-21sed08:20:153
589799118,2cyclictest6234-21grep10:50:263
589799118,2cyclictest4742-21hostname08:15:113
589799118,2cyclictest3807-21ssh09:31:403
589799118,2cyclictest30273-21ssh09:47:523
589799118,2cyclictest29626-21ntp_kernel_pll_10:45:213
589799118,2cyclictest24937-21grep08:55:193
589799118,2cyclictest23506-21ssh11:59:393
589799118,2cyclictest21109-21ssh10:01:283
589799118,2cyclictest20521-21grep07:40:153
589799118,2cyclictest19823-21grep08:45:173
589799118,2cyclictest1938-21head09:50:153
589799118,2cyclictest14616-21grep10:55:273
589799118,2cyclictest12126-21grep07:20:233
589799112,8cyclictest8119-21sensors11:30:253
589799111,6cyclictest0-21swapper/307:25:133
589499119,1cyclictest11622-21tail07:20:182
589499118,2cyclictest5323-21ssh10:50:192
589499118,2cyclictest28778-21tr07:55:232
589499118,2cyclictest20124-21cron07:40:002
589499118,2cyclictest18422-21rm11:55:392
589499118,2cyclictest17581-21tr12:15:152
589499118,2cyclictest14883-21expr08:35:142
589499118,2cyclictest10515-21chrt07:18:542
589499114,3cyclictest225-21systemd-journal09:53:512
589499111,5cyclictest311rcuc/211:40:162
589499111,1cyclictest141rcu_preempt08:20:222
589299118,2cyclictest7888-21cut11:50:151
589299118,2cyclictest32213-21ssh10:27:411
589299118,2cyclictest31577-21sed08:00:261
589299118,2cyclictest29230-21sed10:45:181
589299118,2cyclictest25178-21sort12:00:161
589299118,2cyclictest23903-21grep11:40:161
589299118,2cyclictest21610-21wget08:50:121
589299118,2cyclictest15799-21egrep07:30:161
589299118,2cyclictest13474-21grep10:55:181
589299118,2cyclictest10638-21perf07:20:001
589299117,2cyclictest17582-21sed08:40:171
589299111,2cyclictest225-21systemd-journal07:40:001
588799119,1cyclictest4928-21ssh10:50:150
588799119,1cyclictest22348-21head08:50:150
588799119,1cyclictest16777-21grep09:20:170
588799118,2cyclictest6401-21grep08:15:240
588799118,2cyclictest6117-21grep11:10:160
588799118,2cyclictest5351-21ssh09:32:540
588799118,2cyclictest4499-21ssh12:26:060
588799118,2cyclictest31069-21awk09:10:000
588799118,2cyclictest28846-21basename11:05:000
588799118,2cyclictest23261-21ssh11:40:130
588799118,2cyclictest17250-21ls09:40:140
588799118,2cyclictest17192-21cut11:55:190
588799118,2cyclictest16714-21sed11:35:250
588799118,2cyclictest16714-21sed11:35:250
588799117,3cyclictest5882-21cyclictest09:52:450
588799110,10cyclictest31075-21wc11:05:260
589799107,2cyclictest29865-21uname09:05:193
589799107,2cyclictest23416-21sort11:40:143
589799107,2cyclictest18874-21cat07:35:183
589499108,1cyclictest80792sleep207:13:132
589499108,1cyclictest2293-21cut08:10:142
589499108,1cyclictest176632sleep207:33:562
589499107,2cyclictest27177-21chrt07:53:562
589499107,2cyclictest22791-21grep08:50:192
589499107,2cyclictest22242-21tr11:00:182
589499107,2cyclictest20322-21chrt08:45:202
589499106,3cyclictest32-21ksoftirqd/207:45:172
589499101,8cyclictest26283-21munin-run09:00:002
589299108,1cyclictest29556-21taskset09:05:171
589299107,2cyclictest31048-21diskmemload12:25:131
589299107,2cyclictest28558-21chrt09:03:431
589299100,9cyclictest21554-21ntpq07:40:211
588799108,2cyclictest11230-21ssh09:35:590
588799108,1cyclictest9493-21cat07:15:200
588799108,1cyclictest6156-21cut07:10:140
588799108,1cyclictest3958-21sed10:30:170
588799108,1cyclictest25366-21tr09:45:160
588799108,1cyclictest15342-21grep11:15:240
588799108,1cyclictest11735-21ssh09:55:380
588799108,1cyclictest10562-21basename10:15:090
588799108,1cyclictest101592sleep011:50:430
588799108,1cyclictest0-21swapper/010:35:130
588799107,2cyclictest9820-21ssh12:30:050
588799107,2cyclictest9242-21wc08:25:000
588799107,2cyclictest829-21cat09:10:250
588799107,2cyclictest26316-21grep12:20:160
588799107,2cyclictest20641-21wc10:20:250
588799107,2cyclictest20348-21ssh10:40:150
588799107,2cyclictest17933-21chrt12:15:160
588799105,4cyclictest0-21swapper/010:45:400
588799101,1cyclictest0-21swapper/008:45:150
588799100,9cyclictest0-21swapper/007:25:130
58979997,1cyclictest1641-21taskset08:05:353
58949997,1cyclictest9441-21chrt08:25:092
58929997,1cyclictest20399-21grep08:45:221
58879998,1cyclictest27213-21ssh10:25:080
58879997,2cyclictest12671-21grep08:30:170
58879997,1cyclictest9487-21taskset08:25:160
58879997,1cyclictest2592-21expr08:10:150
58879996,2cyclictest28255-21grep09:00:270
58879996,2cyclictest25147-21basename07:50:120
58879996,2cyclictest16788-21taskset08:40:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional