You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-19 - 14:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue May 19, 2026 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18719210188,9sleep00-21swapper/019:07:300
1869729870,9sleep30-21swapper/319:07:123
1881728676,6sleep10-21swapper/119:08:471
1869128572,9sleep20-21swapper/219:07:072
191002680,0sleep30-21swapper/321:52:533
39682650,0sleep00-21swapper/021:44:080
1902799322,29cyclictest3410-21kworker/3:023:19:033
19022992926,2cyclictest25-21ksoftirqd/121:40:001
1902599270,22cyclictest0-21swapper/223:07:472
1902299265,15cyclictest25-21ksoftirqd/100:13:351
19027992320,3cyclictest17520-21kworker/3:100:29:233
19027992320,2cyclictest3410-21kworker/3:023:25:333
19025992320,2cyclictest55750irq/128-enp2s0-23:55:022
19025992320,2cyclictest55750irq/128-enp2s0-23:55:012
19025992317,5cyclictest0-21swapper/221:45:162
1902299230,23cyclictest0-21swapper/123:54:451
1902099230,22cyclictest0-21swapper/021:32:190
1902799223,18cyclictest11577-21diskmemload23:40:093
19027992219,2cyclictest3410-21kworker/3:023:01:233
1902599220,22cyclictest0-21swapper/221:25:012
1902299225,8cyclictest25-21ksoftirqd/123:13:431
1902299222,4cyclictest15640-21ssh23:08:091
1902299220,1cyclictest0-21swapper/122:19:501
19027992119,2cyclictest8919-21kworker/3:221:18:233
19027992119,2cyclictest3410-21kworker/3:023:11:433
19027992119,2cyclictest3410-21kworker/3:022:57:423
19027992119,2cyclictest17520-21kworker/3:100:11:023
19022992121,0cyclictest0-21swapper/122:50:011
1902099210,20cyclictest0-21swapper/022:26:340
1902099210,0cyclictest0-21swapper/000:08:450
19027992018,2cyclictest8919-21kworker/3:221:11:523
19027992018,2cyclictest3410-21kworker/3:023:46:323
19025992019,1cyclictest0-21swapper/222:17:012
1902599200,2cyclictest0-21swapper/222:12:372
1902599200,20cyclictest0-21swapper/223:39:032
1902599200,0cyclictest0-21swapper/200:02:292
1902299203,2cyclictest25-21ksoftirqd/122:59:421
19022992019,1cyclictest0-21swapper/123:56:261
19022992019,1cyclictest0-21swapper/123:56:261
19022992017,2cyclictest25-21ksoftirqd/123:35:111
1902299200,19cyclictest0-21swapper/121:11:341
1902099200,3cyclictest0-21swapper/000:26:300
1902599190,19cyclictest0-21swapper/221:32:112
1902599190,19cyclictest0-21swapper/200:23:022
1902599190,18cyclictest0-21swapper/223:48:002
1902299192,16cyclictest241rcuc/100:18:411
19022991918,1cyclictest241rcuc/123:29:191
19027991815,3cyclictest8467-21apache222:11:083
19025991810,2cyclictest23772-21sort19:20:002
1902599180,2cyclictest0-21swapper/223:27:032
1902599180,1cyclictest0-21swapper/223:44:122
1902299189,1cyclictest141rcu_preempt22:36:061
19022991817,1cyclictest0-21swapper/121:34:021
19022991816,2cyclictest25-21ksoftirqd/122:07:181
19022991812,6cyclictest25-21ksoftirqd/123:47:261
1902299180,1cyclictest0-21swapper/100:38:521
196812170,0chrt0-21swapper/221:53:202
1902799175,11cyclictest1370-21apache221:30:083
19025991710,2cyclictest9718-21grep21:05:152
1902599170,2cyclictest0-21swapper/222:40:262
1902599170,16cyclictest0-21swapper/221:13:302
1902299178,1cyclictest141rcu_preempt22:34:251
1902299173,13cyclictest25-21ksoftirqd/122:03:581
19022991713,4cyclictest25-21ksoftirqd/123:22:241
19022991710,6cyclictest11577-21diskmemload21:50:461
19027991615,1cyclictest0-21swapper/300:08:533
19027991612,3cyclictest55650irq/127-enp2s0-21:24:093
1902799160,15cyclictest0-21swapper/321:45:503
1902599162,14cyclictest0-21swapper/223:16:412
1902599160,1cyclictest0-21swapper/200:07:222
19022991614,2cyclictest38150irq/131-snd_hda19:25:131
19022991610,5cyclictest11577-21diskmemload23:42:571
1902299160,2cyclictest0-21swapper/123:00:331
1902299160,15cyclictest11577-21diskmemload21:42:571
1902099162,7cyclictest6755-21munin-run21:00:000
1902099160,14cyclictest0-21swapper/023:24:130
1902599159,1cyclictest141rcu_preempt23:03:462
1902599150,2cyclictest0-21swapper/222:08:332
1902599150,14cyclictest0-21swapper/223:50:282
1902599150,14cyclictest0-21swapper/221:39:062
1902299158,3cyclictest38150irq/131-snd_hda21:20:171
1902299156,8cyclictest241rcuc/100:06:271
1902299152,1cyclictest25-21ksoftirqd/100:00:211
19022991514,1cyclictest25-21ksoftirqd/122:42:081
19022991510,4cyclictest11577-21diskmemload23:31:021
1902099157,5cyclictest22313-21apt-get21:55:130
1902099152,1cyclictest0-21swapper/023:00:200
1902099151,7cyclictest2016-21munin-run20:50:000
1902099150,1cyclictest0-21swapper/022:05:430
1902799145,3cyclictest8248-21munin-run22:45:003
19027991411,2cyclictest8698-21grep21:00:283
1902799140,1cyclictest0-21swapper/320:55:123
1902799140,14cyclictest0-21swapper/300:36:023
19025991410,2cyclictest9274-21grep22:45:152
1902299145,8cyclictest15028-21apt-get22:10:141
1902299145,8cyclictest14728-21ssh00:25:101
19022991413,1cyclictest241rcuc/121:26:371
19022991413,1cyclictest241rcuc/120:10:011
19022991410,3cyclictest27466-21grep20:35:161
1902299140,2cyclictest0-21swapper/121:57:141
1902299140,14cyclictest0-21swapper/100:21:081
1902099146,4cyclictest4374-21perf00:00:010
1902099146,4cyclictest4374-21perf00:00:010
1902099142,6cyclictest12779-21apt-get00:05:000
1902099141,1cyclictest141rcu_preempt21:35:330
19020991410,2cyclictest28314-21cat20:35:200
1902099140,0cyclictest0-21swapper/022:02:580
1902799139,3cyclictest14672-21sed20:05:263
1902799139,2cyclictest17322-21grep22:30:213
19027991312,1cyclictest0-21swapper/320:50:133
19027991310,2cyclictest801-21cron19:40:003
19027991310,2cyclictest7140-21grep22:25:013
19027991310,2cyclictest30498-21wc23:55:223
19027991310,2cyclictest30498-21wc23:55:213
19027991310,2cyclictest2524-21wc19:40:273
19027991310,2cyclictest25162-21chrt00:30:263
19027991310,2cyclictest24682-21grep19:20:163
19027991310,2cyclictest24222-21sed20:25:253
19027991310,2cyclictest21332-21ntpq20:20:183
19027991310,2cyclictest21332-21ntpq20:20:183
19027991310,2cyclictest10183-21grep21:05:183
19025991312,1cyclictest0-21swapper/222:24:212
19025991312,1cyclictest0-21swapper/222:01:182
19025991310,2cyclictest293862sleep200:33:462
19025991310,2cyclictest27857-21cat19:25:252
19025991310,2cyclictest18544-21tr20:15:162
19025991310,2cyclictest16141-21basename22:30:142
1902299139,3cyclictest9993-21grep19:55:271
1902299139,3cyclictest7202-21cut21:45:261
1902299137,3cyclictest13490-21ssh22:47:221
1902299133,2cyclictest225-21systemd-journal20:45:011
19022991311,1cyclictest38150irq/131-snd_hda20:25:201
19022991310,3cyclictest11577-21diskmemload21:19:131
19022991310,2cyclictest19993-21head19:10:161
19022991310,1cyclictest241rcuc/120:21:141
19022991310,1cyclictest241rcuc/120:21:131
1902099139,3cyclictest31872-21head00:15:220
1902099139,3cyclictest17278-21wc22:30:200
1902099139,2cyclictest21322-21tr20:20:180
1902099139,2cyclictest21322-21tr20:20:170
1902099131,11cyclictest18531-21grep20:15:150
19020991310,2cyclictest27637-21wc19:25:210
19020991310,2cyclictest21020-21grep23:30:220
19020991310,2cyclictest20850-21sed19:10:260
1902099130,7cyclictest12581-21perf21:29:590
1902099130,0cyclictest0-21swapper/000:23:330
1902799129,2cyclictest8790-21sort22:25:193
1902799129,2cyclictest4766-21sed23:20:273
1902799129,2cyclictest31603-21grep21:40:253
1902799129,2cyclictest28393-21ssh21:39:243
1902799129,2cyclictest21852-21tr23:50:203
1902799129,2cyclictest19209-21sed22:50:263
1902799129,2cyclictest16148-21tail20:10:173
1902799129,2cyclictest14263-21idleruntime-cro00:25:003
1902799129,2cyclictest10681-21grep23:05:143
1902799128,3cyclictest665-21sed20:45:163
1902799128,2cyclictest27432-21grep20:35:163
1902799127,3cyclictest12941-21ls20:05:013
1902599129,2cyclictest8272-21tr21:00:242
1902599129,2cyclictest29946-21cron00:15:002
1902599129,2cyclictest22397-21grep21:55:142
1902599129,2cyclictest18836-21grep23:10:142
1902599129,2cyclictest12093-21cat20:00:252
19025991210,1cyclictest157302sleep200:25:162
1902599120,4cyclictest225-21systemd-journal23:21:582
1902599120,11cyclictest16211-21sed20:10:172
1902599120,0cyclictest0-21swapper/221:42:572
1902299129,2cyclictest817-21dump-pmu-power19:40:001
1902299129,2cyclictest4370-21cron20:55:011
1902299129,2cyclictest28680-21sh00:33:121
1902299129,2cyclictest19969-21cat20:20:011
19022991211,1cyclictest0-21swapper/122:25:411
19022991210,1cyclictest0-21swapper/123:15:151
1902099129,3cyclictest8455-21apache223:36:320
1902099129,2cyclictest32712-21wc00:35:200
1902099129,2cyclictest26882-21ssh22:55:220
1902099129,2cyclictest25048-21dump-pmu-power22:55:000
1902099129,2cyclictest24790-21grep22:15:230
1902099129,2cyclictest12246-21grep21:10:140
1902099129,2cyclictest10397-21sed22:45:210
1902099128,3cyclictest9198-21cron21:04:590
1902099128,3cyclictest10989-21sed23:05:170
1902099128,3cyclictest0-21swapper/022:21:170
1902099124,6cyclictest22951-21taskset00:30:030
1902099120,8cyclictest0-21swapper/020:30:130
1902099120,6cyclictest0-21swapper/022:13:210
1902099120,11cyclictest10431-21wc21:05:200
1902099120,11cyclictest0-21swapper/019:45:130
1902799118,2cyclictest8390-21ssh00:01:373
1902799118,2cyclictest8188-21ssh21:26:373
1902799118,2cyclictest737-21ssh00:15:593
1902799118,2cyclictest561-21ssh22:00:403
1902799118,2cyclictest3814-21ls19:45:163
1902799118,2cyclictest29400-21gzip23:35:203
1902799118,2cyclictest26730-21expr19:25:153
1902799118,2cyclictest25953-21ssh22:15:543
1902799118,2cyclictest22358-21sed19:15:183
1902799118,2cyclictest21211-21ssh23:30:243
1902799118,2cyclictest19581-21cat19:10:143
1902799118,2cyclictest14769-21ssh22:48:243
1902799118,2cyclictest11782-21ssh22:07:533
1902799117,3cyclictest24695-21tr22:35:153
1902799117,3cyclictest23465-21ssh21:55:193
1902799117,3cyclictest18065-21grep20:15:143
1902799116,3cyclictest24959-21apt-get20:30:123
1902599119,1cyclictest3751-21tr19:45:152
1902599119,1cyclictest31697-21grep00:35:142
1902599118,2cyclictest9035-21head19:55:182
1902599118,2cyclictest6752-21ls21:00:002
1902599118,2cyclictest6133-21grep19:50:152
1902599118,2cyclictest31292-21head00:15:182
1902599118,2cyclictest29045-21cut19:30:142
1902599118,2cyclictest28092-21ssh22:36:452
1902599118,2cyclictest2548-21grep20:50:152
1902599118,2cyclictest25443-21ssh23:33:282
1902599118,2cyclictest21959-21ssh21:15:242
1902599115,5cyclictest798-21cron19:40:002
1902599115,4cyclictest27297-21/usr/sbin/munin20:35:182
1902599111,3cyclictest22864-21ssh22:53:182
1902599110,10cyclictest8145-21df_abs22:25:152
1902299119,1cyclictest1541-21grep20:45:251
1902299118,2cyclictest11569-21cron21:10:001
1902299114,6cyclictest8467-21apache222:20:191
1902299110,7cyclictest3206-21cron19:45:001
1902299110,10cyclictest26994-21taskset20:34:171
1902099118,2cyclictest7465-21sed19:50:260
1902099118,2cyclictest5273-21sh21:45:010
1902099118,2cyclictest4485-21ssh22:42:060
1902099118,2cyclictest3728-21ssh23:40:070
1902099118,2cyclictest32619-21wc19:35:310
1902099118,2cyclictest31284-21ssh22:39:150
1902099118,2cyclictest30077-21wc19:30:230
1902099118,2cyclictest30020-21sed21:20:200
1902099118,2cyclictest25160-21ssh23:52:270
1902099118,2cyclictest20133-21ssh21:53:420
1902099118,2cyclictest18802-21tr23:10:140
1902099118,2cyclictest13724-21ssh23:45:250
1902099118,2cyclictest13123-21wget20:05:140
1902099118,2cyclictest11538-21cut23:25:130
1902099117,3cyclictest21316-21cut21:15:180
1902099110,8cyclictest25649-21grep19:20:250
1902099110,10cyclictest32617-21rm23:18:220
1902799108,1cyclictest5817-21taskset19:50:093
1902799108,1cyclictest30534-21chrt19:30:563
1902799108,1cyclictest10116-21chrt19:56:363
1902799107,2cyclictest31681-21chrt20:40:293
1902599108,1cyclictest27076-21munin-run20:35:002
1902599108,1cyclictest25405-21ssh22:55:052
1902599108,1cyclictest1492-21ls20:45:252
1902599107,2cyclictest32041-21chrt20:45:012
1902599107,2cyclictest31700-21rm21:21:092
1902599107,2cyclictest2413-21rt-features19:40:242
1902599107,2cyclictest21188-21taskset19:13:142
1902599107,2cyclictest20943-21grep20:20:142
1902599107,2cyclictest20943-21grep20:20:132
1902299108,1cyclictest7424-21cat21:00:161
1902299108,1cyclictest21565-21taskset19:15:021
1902299107,2cyclictest6882-21expr19:50:201
1902299107,2cyclictest29196-21ls19:30:151
1902299107,2cyclictest10913-21tr20:00:131
1902299104,6cyclictest0-21swapper/119:20:121
1902299100,9cyclictest15405-21basename20:10:111
1902299100,1cyclictest0-21swapper/120:55:171
1902099108,1cyclictest23212-21unixbench-2d19:15:280
1902099107,2cyclictest9063-21head19:55:180
1902099107,2cyclictest22934-21ssh00:10:170
1902099107,2cyclictest2200-21grep19:40:220
1902099107,2cyclictest11509-21basename20:00:160
1902099105,2cyclictest225-21systemd-journal20:51:550
1902099100,9cyclictest153832sleep020:10:010
1902099100,6cyclictest30683-21sed20:40:170
190259997,2cyclictest24209-21unixbench_multi20:25:252
190259997,1cyclictest24002-21apache_processe19:20:122
190259997,1cyclictest13098-21aten2_r7power_e20:05:142
190229996,2cyclictest4706-21tr19:45:221
190209996,2cyclictest22903-21expr20:25:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional