You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 01:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Fri Aug 28, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
857828774,9sleep00-21swapper/007:05:500
889128360,18sleep10-21swapper/107:09:531
882028370,9sleep30-21swapper/307:08:573
680128062,16sleep20-21swapper/207:05:122
264142730,1sleep2489-21cron11:40:002
274772500,0sleep20-21swapper/210:22:462
8991992522,2cyclictest16317-21head08:30:181
899199250,23cyclictest0-21swapper/111:44:101
898999250,2cyclictest0-21swapper/010:10:400
899199221,3cyclictest1206-21kworker/1:209:49:011
898999228,7cyclictest15552-21ssh10:55:000
898999220,1cyclictest0-21swapper/010:21:190
899699210,1cyclictest0-21swapper/311:00:133
899699210,0cyclictest0-21swapper/311:37:583
899499210,21cyclictest0-21swapper/211:49:002
899199210,3cyclictest0-21swapper/111:36:181
898999216,7cyclictest23296-21munin-run07:39:590
899699192,2cyclictest141rcu_preempt10:08:203
8996991912,4cyclictest0-21swapper/311:57:083
899199190,2cyclictest0-21swapper/109:13:151
898999190,4cyclictest0-21swapper/012:24:590
898999190,2cyclictest0-21swapper/012:39:490
898999190,2cyclictest0-21swapper/009:28:490
898999190,1cyclictest0-21swapper/011:59:270
898999190,1cyclictest0-21swapper/010:39:380
8996991810,7cyclictest15851-21apache211:22:033
899499180,18cyclictest0-21swapper/210:33:492
899199181,14cyclictest0-21swapper/109:52:471
898999180,1cyclictest0-21swapper/009:17:270
898999180,17cyclictest0-21swapper/009:47:320
8991991716,1cyclictest38150irq/131-snd_hda09:25:561
899199170,16cyclictest0-21swapper/110:12:401
8996991614,1cyclictest8983-21cyclictest09:28:083
8996991611,3cyclictest4198-21grep09:10:253
899699160,0cyclictest0-21swapper/311:46:343
8991991614,2cyclictest25-21ksoftirqd/109:32:551
898999161,9cyclictest0-21swapper/008:50:150
898999160,14cyclictest0-21swapper/011:14:450
898999160,11cyclictest0-21swapper/012:32:510
243222160,0chrt0-21swapper/211:19:132
899699153,7cyclictest29718-21apt-get09:00:143
8996991514,1cyclictest0-21swapper/310:04:553
899699150,2cyclictest0-21swapper/309:49:423
899499150,14cyclictest0-21swapper/211:52:322
899499150,14cyclictest0-21swapper/209:48:062
899199150,10cyclictest0-21swapper/111:17:321
898999153,6cyclictest7754-21munin-run08:15:000
898999152,2cyclictest225-21systemd-journal12:10:010
8989991511,3cyclictest30703-21wc09:00:200
898999150,12cyclictest0-21swapper/012:25:560
899699149,3cyclictest15394-21tr10:35:143
8996991412,2cyclictest0-21swapper/310:24:113
8996991410,2cyclictest8500-21cut10:50:173
8996991410,2cyclictest5719-21ssh09:50:203
8996991410,2cyclictest29069-21sed07:50:163
899699140,13cyclictest15981-21cut08:30:153
8994991411,2cyclictest25318-21awk07:40:262
8994991410,3cyclictest24006-21grep08:45:262
8994991410,3cyclictest23041-21munin-plugin-st10:40:012
899499140,13cyclictest0-21swapper/210:43:362
8991991411,2cyclictest0-21swapper/111:02:581
899199140,1cyclictest141rcu_preempt09:19:031
899199140,13cyclictest0-21swapper/111:12:341
898999144,9cyclictest16182-21idleruntime-cro07:25:000
8989991411,2cyclictest29878-21wc12:00:250
8989991410,3cyclictest28765-21wc08:55:250
8989991410,3cyclictest28765-21wc08:55:240
898999140,1cyclictest0-21swapper/010:27:300
898999140,1cyclictest0-21swapper/009:39:010
899699139,3cyclictest10718-21wget09:15:143
899699138,3cyclictest30044-21ssh12:00:263
899699133,9cyclictest24031-21munin-node10:33:223
8996991310,2cyclictest5936-21tr10:10:123
8996991310,2cyclictest26441-21grep07:45:153
8996991310,2cyclictest23709-21grep07:40:123
8996991310,2cyclictest22333-21tr12:35:143
8996991310,2cyclictest21404-21grep07:35:143
8996991310,2cyclictest20978-21sort09:59:593
8996991310,2cyclictest16075-21grep10:15:283
8996991310,2cyclictest14512-21grep12:30:163
8996991310,2cyclictest14362-21grep12:10:253
8996991310,1cyclictest0-21swapper/312:08:433
899699130,1cyclictest0-21swapper/312:24:523
899699130,1cyclictest0-21swapper/311:07:503
899499139,3cyclictest32414-21sed07:55:242
899499139,3cyclictest2903-21wc08:05:002
899499138,2cyclictest18947-21ssh09:20:092
899499137,2cyclictest14096-21ntpq09:55:192
899499136,3cyclictest225-21systemd-journal10:30:012
8994991311,2cyclictest32-21ksoftirqd/209:14:272
8994991311,1cyclictest975-21sh12:02:592
8994991310,2cyclictest29857-21cut10:05:132
8994991310,2cyclictest29446-21basename09:00:012
8994991310,2cyclictest29446-21basename09:00:012
8994991310,2cyclictest18684-21cat07:30:012
899499130,1cyclictest0-21swapper/211:33:012
899499130,13cyclictest0-21swapper/212:32:132
899199138,4cyclictest22577-21ssh10:58:481
899199131,1cyclictest1787-21diskmemload11:59:181
8991991311,2cyclictest0-21swapper/110:04:261
8991991311,2cyclictest0-21swapper/109:39:281
8991991310,2cyclictest31407-21grep07:55:151
899199130,2cyclictest0-21swapper/110:23:461
899199130,13cyclictest0-21swapper/109:56:441
899199130,12cyclictest4580-21fgrep08:05:221
898999139,3cyclictest4675-21wc08:05:250
898999139,3cyclictest2365-21head08:00:260
898999139,3cyclictest20829-21cat10:00:000
898999139,3cyclictest20623-21ssh09:20:240
898999139,2cyclictest19437-21grep11:35:170
898999139,2cyclictest11288-21sh09:35:000
898999134,6cyclictest21885-21ssh12:15:200
898999133,8cyclictest19314-21users08:35:270
898999131,8cyclictest1142-21runrttasks09:05:120
8989991310,2cyclictest31964-21grep10:45:130
8989991310,2cyclictest28061-21cat09:45:000
8989991310,2cyclictest25260-21grep07:40:250
8989991310,2cyclictest21426-21wc08:40:260
8989991310,2cyclictest14264-21wc08:25:270
8989991310,2cyclictest13016-21wc07:15:250
8989991310,2cyclictest12938-21cat11:50:280
898999130,2cyclictest0-21swapper/010:19:520
898999130,1cyclictest0-21swapper/011:31:140
898999130,12cyclictest0-21swapper/010:56:230
898999130,0cyclictest0-21swapper/009:51:080
899699129,2cyclictest26977-21cut11:40:123
899699129,2cyclictest24378-21grep10:40:173
899699129,2cyclictest22590-21grep08:45:153
899699129,2cyclictest20658-21sort08:40:163
899699129,2cyclictest20410-21sed07:30:243
899699129,2cyclictest17663-21grep07:25:223
899699129,2cyclictest14668-21sort07:20:163
899699129,2cyclictest12118-21head11:30:253
899699128,3cyclictest4306-21ssh09:30:153
899699128,3cyclictest12489-21sort09:35:143
899699128,2cyclictest13459-21cut08:25:183
899699123,1cyclictest141rcu_preempt12:26:373
899499129,2cyclictest7730-21cron08:15:002
899499129,2cyclictest6040-21ssh09:50:232
899499129,2cyclictest30417-21ssh11:03:242
899499129,2cyclictest20424-21sed11:55:172
899499129,2cyclictest10843-21grep11:10:262
8994991211,1cyclictest0-21swapper/212:08:102
899499120,11cyclictest30189-21cut12:20:162
899199129,2cyclictest8982-21ls10:50:201
899199129,2cyclictest6406-21grep08:10:171
899199129,2cyclictest31387-21sed09:00:261
899199129,2cyclictest3137-21ssh11:25:201
899199129,2cyclictest31042-21grep10:05:201
899199129,2cyclictest28277-21cat08:55:181
899199129,2cyclictest28277-21cat08:55:171
899199129,2cyclictest22158-21grep09:40:251
899199129,2cyclictest19689-21cron08:40:001
899199129,2cyclictest18053-21grep07:25:261
899199129,2cyclictest13130-21grep11:50:281
899199129,2cyclictest11042-21cut11:30:171
899199128,3cyclictest24832-21ssh09:23:251
899199127,4cyclictest20333-21cron12:15:001
899199127,4cyclictest1520-21ssh10:45:281
899199124,7cyclictest8129-21kworker/1:211:49:311
8991991210,1cyclictest4356-21grep12:05:141
899199120,9cyclictest0-21swapper/107:40:131
898999129,2cyclictest26652-21head11:20:160
898999129,2cyclictest25858-21grep11:00:180
898999129,2cyclictest23840-21grep10:40:150
898999129,2cyclictest23127-21cut08:45:170
898999129,2cyclictest17134-21tail07:25:170
898999129,2cyclictest15930-21grep08:30:150
898999128,3cyclictest23177-21sed10:00:250
898999122,6cyclictest0-21swapper/010:30:120
899699119,1cyclictest4008-21cut08:05:173
899699118,2cyclictest9753-21grep11:10:173
899699118,2cyclictest9119-21cut08:15:193
899699118,2cyclictest7623-21ssh11:28:253
899699118,2cyclictest6825-21ls08:10:213
899699118,2cyclictest5025-21ssh10:29:023
899699118,2cyclictest31900-21cut07:55:213
899699118,2cyclictest31637-21ssh10:45:033
899699118,2cyclictest27985-21tune2fs08:55:163
899699118,2cyclictest27985-21tune2fs08:55:163
899699118,2cyclictest2498-21chrt08:01:313
899699118,2cyclictest21927-21grep09:40:253
899699118,2cyclictest21170-21tr12:15:163
899699118,2cyclictest20025-21ssh09:20:173
899699118,2cyclictest19258-21sed11:15:283
899699118,2cyclictest17695-21grep08:35:143
899699118,2cyclictest16131-21ssh10:55:043
899699118,2cyclictest14575-21ssh11:51:323
899699118,2cyclictest11115-21chrt07:12:263
899699118,2cyclictest10672-21grep08:20:143
899499119,1cyclictest8294-21grep08:15:132
899499119,1cyclictest12701-21grep12:10:142
899499118,2cyclictest8132-21ssh10:10:452
899499118,2cyclictest7555-21ssh12:25:362
899499118,2cyclictest5506-21sshd09:30:282
899499118,2cyclictest3436-21rm11:25:232
899499118,2cyclictest31726-21ssh10:45:072
899499118,2cyclictest30477-21grep09:00:172
899499118,2cyclictest29424-21date10:05:002
899499118,2cyclictest27469-21ssh09:44:352
899499118,2cyclictest24096-21cron10:59:592
899499118,2cyclictest22305-21ls12:35:132
899499118,2cyclictest21022-21ssh12:15:152
899499118,2cyclictest20117-21wget08:40:132
899499118,2cyclictest19450-21grep07:30:152
899499118,2cyclictest19234-21sed08:35:262
899499118,2cyclictest16391-21ssh09:37:252
899499118,2cyclictest14704-21ssh09:17:052
899199119,1cyclictest2799-21dpkg08:05:001
899199118,2cyclictest9774-21sed07:10:141
899199118,2cyclictest8711-21grep08:15:161
899199118,2cyclictest7904-21cat10:30:171
899199118,2cyclictest5131-21taskset10:29:081
899199118,2cyclictest29235-21ssh12:20:031
899199118,2cyclictest28497-21expr12:00:151
899199118,2cyclictest28487-21ssh11:20:531
899199118,2cyclictest27422-21ssh12:38:031
899199118,2cyclictest26815-21ssh10:41:251
899199118,2cyclictest25882-21uname08:50:201
899199118,2cyclictest22077-21cat12:15:211
899199118,2cyclictest15902-21tune2fs10:35:161
899199118,2cyclictest14232-21grep07:20:131
899199118,2cyclictest13227-21cat08:25:161
899199117,3cyclictest14397-21grep12:30:161
899199117,3cyclictest1406-21grep11:05:191
899199115,2cyclictest225-21systemd-journal07:20:011
899199110,11cyclictest0-21swapper/110:15:271
898999118,2cyclictest9861-21cut07:10:140
898999118,2cyclictest9001-21wc08:15:190
898999118,2cyclictest7177-21ssh11:09:040
898999118,2cyclictest31702-21tail10:05:260
898999118,2cyclictest27212-21wc07:45:200
898999118,2cyclictest18784-21grep11:15:220
898999116,3cyclictest13-21ksoftirqd/009:10:170
898999113,7cyclictest7660-21ssh11:28:260
898999110,8cyclictest14027-21ssh12:10:220
898999110,5cyclictest0-21swapper/011:46:160
899699108,1cyclictest24894-21gdbus08:50:003
899699107,2cyclictest32570-21taskset09:05:143
899699107,2cyclictest12532-21grep07:15:193
899499108,2cyclictest8242-21ls10:50:152
899499108,2cyclictest13238-21expr08:25:162
899499108,1cyclictest5113-21taskset08:06:532
899499108,1cyclictest277952sleep207:46:472
899499108,1cyclictest15069-21cat07:20:182
899499108,1cyclictest14556-21sh10:15:172
899499108,1cyclictest12336-21tail07:15:172
899499107,2cyclictest9982-21head07:10:152
899499107,2cyclictest28067-21rm11:40:192
899499107,2cyclictest28057-21rm11:20:342
899499107,2cyclictest12408-21taskset08:23:442
899499105,2cyclictest225-21systemd-journal09:09:072
899499102,4cyclictest225-21systemd-journal07:39:562
899499101,2cyclictest0-21swapper/208:50:152
899499100,9cyclictest31760-21ssh09:27:102
899499100,9cyclictest0-21swapper/211:07:322
899199108,1cyclictest5526-21egrep12:25:141
899199108,1cyclictest28288-21seq07:50:101
899199108,1cyclictest22502-21egrep08:45:151
899199108,1cyclictest19950-21chrt08:40:111
899199108,1cyclictest122252sleep108:21:241
899199107,2cyclictest19419-21tune2fs07:30:161
899199107,2cyclictest1214-21sed09:05:251
899199101,7cyclictest38150irq/131-snd_hda07:45:121
898999108,1cyclictest31390-21cut07:55:150
898999108,1cyclictest28936-21chrt11:40:260
898999107,2cyclictest18810-21aten2_r7power_c07:30:130
898999107,2cyclictest12214-21chrt08:21:140
89949998,1cyclictest28528-21grep07:50:142
89949998,1cyclictest16674-21sed08:30:232
89919996,2cyclictest21626-21expr07:35:151
89899996,2cyclictest28360-21grep07:50:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional