You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-09 - 17:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Fri Jan 09, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25311210996,9sleep20-21swapper/207:05:032
2741028674,8sleep00-21swapper/007:09:310
2727228661,9sleep30-21swapper/307:07:453
2616828365,13sleep10-21swapper/107:05:181
2754099270,26cyclictest0-21swapper/211:55:132
2754399236,10cyclictest0-21swapper/307:50:123
27540992110,9cyclictest0-21swapper/209:40:132
2754099210,4cyclictest0-21swapper/207:55:142
2753899210,20cyclictest0-21swapper/109:55:121
27540991914,4cyclictest10970-21ntp_offset09:55:192
2754099187,10cyclictest0-21swapper/212:00:122
27540991811,5cyclictest0-21swapper/212:24:592
2753899180,12cyclictest0-21swapper/108:28:111
2754099179,7cyclictest0-21swapper/209:30:242
2754099179,5cyclictest0-21swapper/210:40:142
2754099172,14cyclictest0-21swapper/209:10:132
27538991717,0cyclictest0-21swapper/111:05:171
2754099167,7cyclictest0-21swapper/209:05:132
2754099167,7cyclictest0-21swapper/209:05:132
2754099164,8cyclictest0-21swapper/208:35:102
2754099163,11cyclictest0-21swapper/208:00:122
2754099159,3cyclictest12521-21head11:05:262
27540991510,4cyclictest17015-21sed09:00:242
27540991510,3cyclictest9485-21wc11:00:192
2753399155,9cyclictest20340-21apt09:10:130
2754399149,3cyclictest5739-21cut07:30:153
2754399149,3cyclictest4336-21cut10:50:173
27543991411,2cyclictest25746-21wc10:25:253
2754399140,1cyclictest0-21swapper/308:45:123
2754399140,12cyclictest11963-21tr10:00:003
2754099149,4cyclictest21053-21grep12:30:262
27540991411,2cyclictest22604-21grep11:30:012
27540991411,2cyclictest17217-21grep11:15:262
27540991411,2cyclictest0-21swapper/207:40:152
2754099141,11cyclictest31341-21pmu-power11:45:212
27540991410,3cyclictest13873-21grep10:00:242
27540991410,3cyclictest13578-21grep12:15:252
27538991411,2cyclictest18680-21grep07:55:271
27538991410,3cyclictest5762-21sh10:55:001
27538991410,3cyclictest29719-21sed08:20:161
2753399145,4cyclictest19097-21dump-pmu-power08:00:000
27533991411,2cyclictest973-21cut09:35:160
27543991311,1cyclictest0-21swapper/309:00:123
27543991310,2cyclictest7734-21grep07:35:143
27543991310,2cyclictest2905-21grep08:30:253
2754399130,1cyclictest0-21swapper/311:40:133
2754399130,12cyclictest9550-21cron09:55:003
2754099139,3cyclictest26676-21wc11:35:232
2754099139,3cyclictest10201-21sort12:10:162
2754099138,2cyclictest25052-21taskset10:25:172
2754099138,1cyclictest0-21swapper/210:30:132
2754099137,5cyclictest0-21swapper/212:35:202
2754099137,5cyclictest0-21swapper/211:30:192
2754099137,5cyclictest0-21swapper/207:25:132
2754099136,6cyclictest0-21swapper/208:20:212
2754099136,5cyclictest0-21swapper/207:45:132
2754099132,7cyclictest0-21swapper/211:10:162
27540991312,1cyclictest0-21swapper/208:05:112
27540991311,1cyclictest0-21swapper/210:35:122
27540991311,1cyclictest0-21swapper/210:20:122
27540991310,2cyclictest7778-21head08:40:252
27540991310,2cyclictest17484-21gltestperf12:25:182
27540991310,2cyclictest16716-21idleruntime-cro07:55:002
2754099130,7cyclictest225-21systemd-journal10:57:532
2753899139,3cyclictest20927-21sort11:25:161
2753899138,3cyclictest31368-21head07:15:211
2753899138,3cyclictest14414-21grep11:10:221
27538991310,2cyclictest3168-21head11:55:171
2753899130,10cyclictest285302sleep108:18:111
27533991310,2cyclictest31199-21cat09:30:200
27533991310,2cyclictest30551-21grep10:35:230
27533991310,2cyclictest2440-21cron07:25:000
27533991310,2cyclictest22486-21cron11:30:000
27533991310,2cyclictest19987-21sort12:30:170
27533991310,2cyclictest16748-21ls11:15:200
2753399130,7cyclictest24919-21basename09:20:000
2753399130,10cyclictest0-21swapper/010:05:120
2754399129,2cyclictest9872-21grep12:10:143
2754399129,2cyclictest27485-21readlink11:40:013
2754399129,2cyclictest23885-21munin-plugin-st12:40:003
2754399129,2cyclictest23307-21df11:30:153
2754399129,2cyclictest20945-21grep12:30:263
2754399129,2cyclictest20115-21grep08:00:163
2754399129,2cyclictest13803-21sort11:10:163
2754399129,2cyclictest13348-21tr10:00:183
2754399129,2cyclictest12522-21sed11:05:263
2754399128,3cyclictest21663-21apache207:10:113
2754399128,3cyclictest17474-21grep07:55:163
2754399128,3cyclictest12919-21cron08:54:593
2754399128,2cyclictest18654-21grep09:05:163
2754399128,2cyclictest18654-21grep09:05:163
2754399120,11cyclictest29079-21grep09:25:233
2754099129,2cyclictest569-21tr09:35:142
2754099129,2cyclictest28714-21wc11:40:192
2754099129,2cyclictest23479-21grep09:15:152
2754099129,2cyclictest18179-21tr11:20:142
2754099129,2cyclictest1707-21grep11:50:242
2754099128,2cyclictest25452-21tr09:20:142
2754099128,1cyclictest9231-21taskset12:08:512
2754099127,4cyclictest0-21swapper/209:50:002
2754099125,6cyclictest17347-21expr10:10:142
27540991210,1cyclictest0-21swapper/208:50:162
27540991210,1cyclictest0-21swapper/208:45:122
2754099120,8cyclictest0-21swapper/207:20:112
2753899129,2cyclictest3447-21grep07:25:171
2753899129,2cyclictest29876-21dump-pmu-power09:30:001
2753899129,2cyclictest2921-21grep08:30:251
2753899129,2cyclictest24974-21wc09:20:001
2753899129,2cyclictest21840-21wc09:10:251
2753899129,2cyclictest20106-21sed12:30:171
2753899129,2cyclictest13812-21grep08:55:161
2753899124,4cyclictest225-21systemd-journal07:25:001
2753899120,9cyclictest0-21swapper/109:20:131
2753399129,2cyclictest8194-21grep07:35:160
2753399129,2cyclictest7687-21tr09:50:140
2753399129,2cyclictest7030-21cat12:05:000
2753399129,2cyclictest4565-21sh11:59:590
2753399129,2cyclictest31377-21grep07:15:210
2753399129,2cyclictest30029-21cat07:15:000
2753399129,2cyclictest28043-21tr11:40:150
2753399129,2cyclictest2716-21cat08:30:240
2753399129,2cyclictest26314-21perf08:15:000
2753399129,2cyclictest23903-21cron10:25:000
2753399129,2cyclictest21507-21awk08:05:000
2753399129,2cyclictest19077-21sh10:15:000
2753399129,2cyclictest11701-21cron12:15:000
2753399129,2cyclictest11537-21head11:05:170
2753399128,3cyclictest6775-21grep08:40:160
2753399128,3cyclictest23457-21sed08:05:240
2753399128,2cyclictest13488-21wc07:45:200
2753399126,2cyclictest225-21systemd-journal12:40:000
2753399120,2cyclictest0-21swapper/011:30:130
2753399120,1cyclictest0-21swapper/011:50:150
2753399120,11cyclictest0-21swapper/010:44:120
2754399118,2cyclictest7238-21cut10:55:213
2754399118,2cyclictest6940-21sshd12:04:533
2754399118,2cyclictest596-21expr07:20:133
2754399118,2cyclictest4348-21chrt11:57:223
2754399118,2cyclictest348-21cat08:25:253
2754399118,2cyclictest3391-21cut07:25:173
2754399118,2cyclictest3252-21df09:40:153
2754399118,2cyclictest32478-21cat09:35:013
2754399118,2cyclictest32291-21wc10:40:183
2754399118,2cyclictest32254-21cron07:20:003
2754399118,2cyclictest322-21grep11:50:133
2754399118,2cyclictest30975-21chrt08:23:503
2754399118,2cyclictest30089-21taskset11:45:053
2754399118,2cyclictest29393-21expr10:35:133
2754399118,2cyclictest27326-21grep08:15:173
2754399118,2cyclictest23032-21grep09:15:133
2754399118,2cyclictest22580-21grep08:05:153
2754399118,2cyclictest22547-21grep10:20:163
2754399118,2cyclictest19302-21taskset10:15:023
2754399118,2cyclictest17578-21taskset11:19:023
2754399118,2cyclictest17239-21df12:25:163
2754399118,2cyclictest14408-21grep08:55:213
2754399118,2cyclictest14248-21chrt07:48:563
2754399118,2cyclictest13875-21taskset12:17:263
2754399118,2cyclictest10594-21expr07:40:163
2754399117,3cyclictest9689-21wc11:00:213
2754399110,10cyclictest22166-21taskset09:10:493
2754099118,2cyclictest8490-21wc07:35:192
2754099118,2cyclictest6809-21head07:30:252
2754099118,2cyclictest32255-21cron07:20:002
2754099118,2cyclictest2927-21sed08:30:252
2754099118,2cyclictest27280-21head08:15:172
2754099115,5cyclictest0-21swapper/208:25:182
2754099114,6cyclictest477-21rs:main2
2754099111,4cyclictest0-21swapper/210:05:272
2754099110,10cyclictest1683-21expr10:45:142
2753899119,1cyclictest8923-21grep12:05:251
2753899118,2cyclictest7947-21df07:35:141
2753899118,2cyclictest7513-21wc10:55:251
2753899118,2cyclictest5219-21grep09:45:121
2753899118,2cyclictest32076-21grep10:40:161
2753899118,2cyclictest31813-21sed11:45:251
2753899118,2cyclictest31378-21grep09:30:221
2753899118,2cyclictest24882-21apt11:35:001
2753899118,2cyclictest1819-21wc09:35:251
2753899118,2cyclictest16774-21wc10:10:001
2753899118,2cyclictest1573-21grep10:45:141
2753899118,2cyclictest14554-21cat07:50:001
2753899118,2cyclictest1341-21wc11:50:201
2753899118,2cyclictest12924-21idleruntime-cro08:55:001
2753899118,2cyclictest12216-21tr12:15:141
2753899118,2cyclictest11084-21head07:40:201
2753899118,2cyclictest10856-21grep12:10:211
2753899118,2cyclictest10468-21taskset08:49:051
2753899117,3cyclictest20334-21ls10:15:181
2753899114,3cyclictest225-21systemd-journal11:20:001
2753899110,9cyclictest0-21swapper/112:35:131
2753899110,10cyclictest0-21swapper/108:05:111
2753399119,1cyclictest7194-21cron07:35:000
2753399119,1cyclictest18432-21grep12:25:270
2753399118,2cyclictest9646-21grep11:00:210
2753399118,2cyclictest4436-21tail08:35:140
2753399118,2cyclictest3083-21taskset10:46:090
2753399118,2cyclictest28255-21df09:25:160
2753399118,2cyclictest26816-21grep11:35:240
2753399118,2cyclictest18724-21tail09:05:160
2753399118,2cyclictest18724-21tail09:05:160
2753399118,2cyclictest16039-21sed12:20:260
2753399118,2cyclictest15301-21apt11:15:000
2753399118,2cyclictest11893-21taskset07:44:110
2753399116,4cyclictest30424-21munin-node11:45:060
2753399113,7cyclictest4833-21perf09:45:000
2753399111,2cyclictest0-21swapper/010:30:130
2754399108,1cyclictest80352sleep308:43:213
2754399108,1cyclictest5284-21grep09:45:123
2754399108,1cyclictest27919-21mailstats10:30:223
2754399108,1cyclictest25133-21chrt09:20:073
2754399108,1cyclictest24663-21expr08:10:143
2754399108,1cyclictest18414-21expr11:20:143
2754399107,2cyclictest7332-21grep12:05:133
2754399107,2cyclictest3793-21cut08:35:113
2754399107,2cyclictest20579-21grep11:25:133
2754399107,2cyclictest17645-21grep10:10:163
2754399107,2cyclictest16074-21grep12:20:263
2754399107,2cyclictest15239-21sort10:05:293
2754399107,2cyclictest1049-21sed09:35:163
2754399100,9cyclictest1667-21expr10:45:143
2754099108,1cyclictest7530-21aten2_r7power_p09:50:142
2754099108,1cyclictest36262sleep210:50:102
2754099107,2cyclictest20498-21sh10:15:202
2754099105,1cyclictest225-21systemd-journal09:27:532
2754099104,5cyclictest29431-21tail07:10:262
2754099104,5cyclictest14290-21needreboot08:55:192
2753899109,1cyclictest25-21ksoftirqd/110:39:351
2753899108,1cyclictest8970-21df_abs11:00:141
2753899108,1cyclictest5268-21expr12:00:141
2753899108,1cyclictest4025-21expr08:35:121
2753899108,1cyclictest27986-21sort07:10:141
2753899108,1cyclictest26859-21cat10:30:141
2753899108,1cyclictest24142-21taskset10:25:091
2753899108,1cyclictest20298-21uname08:00:171
2753899108,1cyclictest16584-21cat12:25:001
2753899108,1cyclictest15655-21cat07:50:191
2753899107,2cyclictest9382-21sshd09:52:531
2753899107,2cyclictest5195-21cut07:30:131
2753899107,2cyclictest3075-21expr09:40:141
2753899107,2cyclictest25168-21wget11:35:121
2753899107,2cyclictest22476-21grep10:20:141
2753899107,2cyclictest18273-21cat12:25:271
2753899107,2cyclictest14174-21taskset10:03:091
2753899106,3cyclictest16292-21head09:00:151
2753899100,9cyclictest28153-21expr11:40:151
2753399108,1cyclictest4780-21taskset07:29:370
2753399108,1cyclictest18305-21expr11:20:140
2753399108,1cyclictest17830-21cat09:05:000
2753399108,1cyclictest14863-21tr07:50:140
2753399108,1cyclictest12481-21expr10:00:130
2753399107,2cyclictest7853-21sort12:05:160
2753399107,2cyclictest6861-21basename10:55:180
2753399107,2cyclictest5210-21cut09:45:120
2753399107,2cyclictest3931-21tr10:50:150
2753399107,2cyclictest29107-21wget08:20:140
2753399107,2cyclictest26862-21ls08:15:140
2753399107,2cyclictest20979-21grep10:15:250
2753399107,2cyclictest15180-21chrt08:58:410
2753399107,2cyclictest13167-21ls12:15:200
2753399107,2cyclictest12137-21wc08:50:200
2753399107,2cyclictest1191-21cat08:30:000
2753399107,2cyclictest10239-21cat09:55:140
2753399100,9cyclictest104662sleep008:49:030
275389997,1cyclictest6472-21expr08:40:141
275389997,1cyclictest197232sleep109:05:431
275389997,1cyclictest197232sleep109:05:431
275389997,1cyclictest16964-21bc10:10:111
275389996,2cyclictest24632-21cat08:10:141
275389990,4cyclictest18603-21cut11:20:161
275339996,2cyclictest25264-21grep10:25:190
275339996,2cyclictest25194-21cut09:20:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional