You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-06 - 06:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Wed May 06, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2623429986,8sleep00-21swapper/019:08:070
2605929064,8sleep20-21swapper/219:05:522
2603828564,16sleep30-21swapper/319:05:373
2424428467,13sleep10-21swapper/119:05:071
2646899294,6cyclictest2305-21gltestperf00:10:170
2646899281,3cyclictest17848-21sendmail00:20:000
2646899271,5cyclictest19902-21diskmemload22:06:060
26468992710,16cyclictest0-21swapper/022:15:200
2647399260,26cyclictest0-21swapper/200:24:412
2646899264,3cyclictest14640-21fschecks_count19:50:160
2646899260,3cyclictest0-21swapper/019:45:140
2646899253,2cyclictest6395-21aten2_r7power_p21:40:150
2646899250,2cyclictest0-21swapper/020:45:130
2647399240,23cyclictest0-21swapper/223:09:412
2647399240,23cyclictest0-21swapper/223:09:402
2647399240,23cyclictest0-21swapper/222:33:112
2647099240,1cyclictest0-21swapper/121:37:471
2646899240,2cyclictest0-21swapper/021:37:500
2647399230,16cyclictest0-21swapper/223:10:492
2647099230,22cyclictest0-21swapper/123:32:061
2646899231,2cyclictest24420-21sed23:45:200
2646899230,4cyclictest0-21swapper/022:26:460
2647399221,21cyclictest0-21swapper/222:51:472
2647899210,21cyclictest0-21swapper/322:57:333
2647099213,17cyclictest19902-21diskmemload22:25:431
2646899211,4cyclictest0-21swapper/000:30:130
2646899211,4cyclictest0-21swapper/000:30:120
2646899210,5cyclictest0-21swapper/023:38:440
2646899210,3cyclictest0-21swapper/020:25:130
26478992019,1cyclictest0-21swapper/322:29:513
26478992010,7cyclictest1089-21apache200:36:583
2647399200,20cyclictest0-21swapper/222:59:122
2647399200,20cyclictest0-21swapper/221:47:182
2647399200,0cyclictest0-21swapper/221:19:482
2647099200,20cyclictest0-21swapper/123:28:321
2647099200,20cyclictest0-21swapper/121:11:081
2646899208,7cyclictest11371-21ssh23:56:190
2646899200,2cyclictest0-21swapper/022:48:140
26478991918,1cyclictest0-21swapper/321:58:483
2647399190,19cyclictest0-21swapper/223:50:422
2647399190,19cyclictest0-21swapper/221:24:332
26470991918,1cyclictest0-21swapper/122:15:141
26470991917,1cyclictest0-21swapper/122:36:331
2647099191,17cyclictest4843-21sed20:35:251
241162190,0sleep30-21swapper/323:45:173
2647899180,17cyclictest0-21swapper/322:41:153
2647399180,3cyclictest0-21swapper/200:01:152
2647399180,2cyclictest0-21swapper/222:23:072
2647399180,18cyclictest0-21swapper/222:15:182
26470991810,1cyclictest0-21swapper/121:51:181
2646899188,8cyclictest11256-21ssh22:40:140
2646899185,12cyclictest19902-21diskmemload23:32:450
2646899184,2cyclictest5132-21apache_accesses23:15:120
26468991811,6cyclictest10599-21wc22:20:220
2646899181,16cyclictest0-21swapper/022:10:130
2646899180,5cyclictest0-21swapper/022:31:470
2646899180,3cyclictest0-21swapper/023:50:170
2646899180,1cyclictest0-21swapper/019:10:140
2647899171,2cyclictest141rcu_preempt23:30:143
2647399173,7cyclictest0-21swapper/221:30:112
2647399170,17cyclictest0-21swapper/222:09:572
26470991716,1cyclictest6009-21sed21:20:191
26470991713,4cyclictest0-21swapper/100:01:241
2647099171,1cyclictest141rcu_preempt21:40:171
2646899175,7cyclictest12555-21awk20:55:000
2646899173,7cyclictest28421-21munin-run23:10:000
2646899173,7cyclictest28421-21munin-run23:10:000
2646899172,5cyclictest14925-21cron21:00:010
2647899163,12cyclictest0-21swapper/323:02:133
26478991611,5cyclictest21809-21sh22:46:013
2647899160,2cyclictest0-21swapper/323:50:143
26473991614,2cyclictest32-21ksoftirqd/221:51:192
2647099169,6cyclictest38150irq/131-snd_hda21:18:491
26470991614,1cyclictest0-21swapper/122:02:221
2647099160,16cyclictest0-21swapper/123:40:561
2647099160,16cyclictest0-21swapper/100:08:541
2647099160,14cyclictest0-21swapper/123:00:111
2646899169,6cyclictest4880-21grep19:30:160
2646899169,6cyclictest22368-21grep21:10:250
2646899168,7cyclictest31166-21chrt21:15:390
2646899168,7cyclictest21819-21ssh00:21:340
2646899168,7cyclictest21517-21sleep23:25:010
2646899168,2cyclictest25897-21chrt00:05:180
2646899168,2cyclictest15440-21ssh23:01:320
2646899167,2cyclictest15935-21rm00:37:240
2646899160,15cyclictest10036-21rm22:39:580
2646899160,0cyclictest0-21swapper/023:29:330
2647899150,1cyclictest0-21swapper/300:15:403
2647899150,0cyclictest0-21swapper/322:34:123
26473991515,0cyclictest0-21swapper/221:27:502
26473991515,0cyclictest0-21swapper/221:27:492
26473991514,1cyclictest0-21swapper/200:37:222
2647399151,1cyclictest0-21swapper/222:45:142
2647399150,2cyclictest0-21swapper/222:39:402
2647399150,15cyclictest0-21swapper/200:25:232
2647399150,13cyclictest28963-21wc19:15:002
2647399150,12cyclictest0-21swapper/221:35:172
26470991513,2cyclictest19902-21diskmemload22:23:531
26470991512,3cyclictest38150irq/131-snd_hda00:20:221
2647099151,14cyclictest9244-21irqstats23:55:181
2646899159,5cyclictest7309-21sed20:40:270
2646899158,6cyclictest23686-21sed21:30:250
2646899158,2cyclictest30296-21taskset00:26:330
2646899158,2cyclictest2229-21ssh21:56:150
2646899158,2cyclictest13655-21tr21:25:140
2646899158,2cyclictest13655-21tr21:25:140
2646899158,1cyclictest31673-21ssh22:52:050
2646899158,1cyclictest31552-21ssh23:10:450
2646899158,1cyclictest3127-21cat20:35:010
2646899157,3cyclictest5384-21sendmail-msp20:39:590
2646899157,2cyclictest23951-21grep21:50:160
2646899157,2cyclictest2271-21cut19:25:160
2646899157,2cyclictest15841-21fschecks_time21:00:160
2646899157,1cyclictest8253-21sed22:00:150
2646899157,1cyclictest27229-21gzip20:15:210
2646899156,2cyclictest29552-21tr20:20:210
2646899154,4cyclictest1181-21basename19:25:010
2646899150,5cyclictest25449-21sshd20:12:350
2646899150,14cyclictest21563-21ssh00:02:380
2646899150,13cyclictest19053-21tr20:00:150
2647899140,1cyclictest0-21swapper/321:37:483
2647899140,1cyclictest0-21swapper/300:20:463
26473991413,1cyclictest0-21swapper/219:49:472
2647399141,1cyclictest0-21swapper/222:14:282
2647399141,13cyclictest0-21swapper/223:48:082
26473991411,2cyclictest13915-21grep23:00:272
26470991413,1cyclictest0-21swapper/123:16:401
26470991411,2cyclictest14568-21wc21:25:201
26470991411,2cyclictest14568-21wc21:25:191
26470991410,4cyclictest38150irq/131-snd_hda22:11:581
2647099140,1cyclictest0-21swapper/122:44:281
2647099140,14cyclictest0-21swapper/100:31:381
2647099140,14cyclictest0-21swapper/100:31:371
2646899148,1cyclictest6830-21test19:35:020
2646899147,1cyclictest17550-21chrt21:46:050
2646899146,2cyclictest4793-21rm22:55:190
2646899146,2cyclictest18724-21sed21:05:200
2646899146,2cyclictest10554-21chrt21:23:140
2647899137,6cyclictest6579-21apache222:38:313
26478991310,2cyclictest20886-21sh23:05:173
26478991310,2cyclictest20886-21sh23:05:163
2647899130,1cyclictest0-21swapper/323:56:043
2647899130,12cyclictest2558-21sed20:30:263
2647399139,3cyclictest17342-21wc21:05:002
2647399132,9cyclictest0-21swapper/222:25:152
26473991311,1cyclictest55650irq/127-enp2s0-00:32:032
26473991311,1cyclictest55650irq/127-enp2s0-00:32:022
26473991310,2cyclictest879-21cat00:10:012
26473991310,2cyclictest8016-21grep19:35:202
26473991310,2cyclictest28048-21systemctl20:19:592
26473991310,2cyclictest1765-21grep00:10:142
2647399130,12cyclictest23601-21grep20:10:142
26470991310,2cyclictest3114-21cat20:35:001
26470991310,2cyclictest19245-21grep22:45:151
26470991310,2cyclictest16655-21tr19:55:141
26470991310,2cyclictest12880-21tr19:45:221
2647099130,1cyclictest0-21swapper/122:58:041
2646899137,2cyclictest31006-21taskset19:17:460
2646899136,1cyclictest23012-21chrt20:06:510
2646899135,3cyclictest16425-21cut19:55:130
2646899132,2cyclictest225-21systemd-journal23:42:170
2647899129,2cyclictest9159-21apache_volume22:20:123
2647899129,2cyclictest8683-21grep20:45:163
2647899129,2cyclictest21522-21awk23:25:013
2647899129,2cyclictest18542-21cat00:00:273
2647899129,2cyclictest18483-21grep21:05:193
2647899125,2cyclictest9273-21apt-get19:40:113
26478991211,1cyclictest0-21swapper/322:16:533
26478991211,1cyclictest0-21swapper/321:11:083
26478991210,1cyclictest3204-21grep00:30:153
26478991210,1cyclictest3204-21grep00:30:143
26478991210,1cyclictest25148-21sed22:10:153
2647899120,1cyclictest0-21swapper/321:30:143
2647399129,2cyclictest6623-21grep21:40:162
2647399129,2cyclictest4830-21grep19:30:172
2647399129,2cyclictest22869-21sed20:05:272
2647399129,2cyclictest19810-21cat20:00:202
2647399129,2cyclictest14799-21grep19:50:172
2647399129,2cyclictest10179-21grep20:49:592
2647399128,3cyclictest3497-21grep20:35:142
2647399124,4cyclictest13576-21/usr/sbin/munin23:20:262
2647399122,6cyclictest0-21swapper/223:30:042
26473991210,2cyclictest32-21ksoftirqd/222:40:172
2647099129,2cyclictest26869-21cut20:15:181
2647099129,2cyclictest22084-21tr21:30:131
2647099129,2cyclictest11684-21grep20:50:211
2647099126,5cyclictest9340-21tr19:40:111
2647099125,2cyclictest141rcu_preempt00:11:031
26470991210,1cyclictest29377-21tr23:10:141
26470991210,1cyclictest28060-21grep22:30:261
26470991210,1cyclictest14743-21tail19:50:171
2647099120,2cyclictest141rcu_preempt22:06:091
2647099120,12cyclictest0-21swapper/119:30:271
2647099120,11cyclictest0-21swapper/122:54:451
2647099120,10cyclictest31601-21ssh21:55:131
2646899122,4cyclictest11105-21sshd19:41:360
2647899119,1cyclictest6372-21grep20:40:173
2647899119,1cyclictest3362-21grep19:25:263
2647899119,1cyclictest26945-21cut19:10:153
2647899119,1cyclictest13148-21chrt21:44:113
2647899119,1cyclictest0-21swapper/300:11:103
2647899118,2cyclictest8981-21ssh23:35:383
2647899118,2cyclictest8823-21cat22:00:193
2647899118,2cyclictest31117-21sed23:10:263
2647899118,2cyclictest27274-21expr00:25:153
2647899118,2cyclictest25040-21grep21:50:273
2647899118,2cyclictest24996-21sh00:05:083
2647899118,2cyclictest19555-21ssh23:42:193
2647899118,2cyclictest17445-21cat19:55:203
2647899118,2cyclictest14333-21cat19:50:153
2647899118,2cyclictest13074-21tr20:55:143
2647899117,1cyclictest19902-21diskmemload21:29:373
2647899117,1cyclictest19902-21diskmemload21:29:363
2647899115,3cyclictest6568-21apache223:25:263
2647899110,10cyclictest10393-21sh21:23:053
2647399119,1cyclictest6060-21hwlatdetect23:15:172
2647399119,1cyclictest31754-21grep19:20:152
2647399119,1cyclictest22483-21ssh21:10:282
2647399119,1cyclictest17166-21cut19:55:162
2647399118,2cyclictest7622-21chrt20:42:572
2647399118,2cyclictest518-21date20:30:012
2647399118,2cyclictest3052-21ssh21:56:512
2647399118,2cyclictest22804-21grep23:25:162
2647399118,2cyclictest16139-21cut23:40:182
2647399118,2cyclictest14502-21grep20:55:262
2647399118,2cyclictest12448-21ssh23:57:092
2647399118,2cyclictest12339-21ssh23:38:142
2647399118,2cyclictest12034-21grep20:50:262
2647399118,2cyclictest11670-21timerandwakeup00:15:262
2647399114,6cyclictest6568-21apache222:00:222
2647099119,1cyclictest29863-21sed19:15:171
2647099118,2cyclictest8647-21chrt19:35:481
2647099118,2cyclictest3842-21ssh23:52:131
2647099118,2cyclictest3683-21chrt19:29:141
2647099118,2cyclictest32508-21wc19:20:201
2647099118,2cyclictest28676-21expr20:20:141
2647099118,2cyclictest28338-21tail19:10:271
2647099118,2cyclictest25768-21rm23:45:361
2647099118,2cyclictest24281-21grep20:10:171
2647099118,2cyclictest22641-21wc20:05:261
2647099118,2cyclictest21742-21cat23:05:221
2647099118,2cyclictest21742-21cat23:05:211
2647099118,2cyclictest20259-21wc20:00:261
2647099118,2cyclictest15857-21cut21:00:161
2647099118,2cyclictest14519-21ssh21:45:051
2647099118,2cyclictest13536-21head20:55:171
2647099118,2cyclictest13246-21sed00:35:261
2647099118,2cyclictest11680-21sort00:15:261
2647099118,2cyclictest10774-21chrt23:37:031
2647099117,3cyclictest27999-21ssh00:25:191
2647099110,10cyclictest16333-21ssh23:20:581
2647899108,2cyclictest29698-21grep19:15:183
2647899108,2cyclictest28753-21expr20:20:153
2647899108,2cyclictest15497-21ssh21:45:173
2647899108,2cyclictest10800-21expr20:50:153
2647899108,1cyclictest32292sleep320:35:123
2647899108,1cyclictest32286-21sh22:52:333
2647899108,1cyclictest28792-21ssh21:15:113
2647899108,1cyclictest23858-21expr20:10:153
2647899108,1cyclictest21169-21aten2_r7power_c20:05:143
2647899107,2cyclictest6906-21cut19:35:133
2647899107,2cyclictest4898-21tune2fs19:30:173
2647899107,2cyclictest4865-21ssh23:15:003
2647899107,2cyclictest15888-21ssh22:05:023
2647899107,2cyclictest15424-21grep21:00:143
2647899107,2cyclictest13350-21grep19:45:273
2647399108,1cyclictest992-21egrep20:30:142
2647399108,1cyclictest92322sleep219:40:052
2647399108,1cyclictest28727-21expr20:20:152
2647399107,2cyclictest18210-21expr21:05:172
2647399100,9cyclictest310102sleep219:17:492
2647099107,2cyclictest30684-21apache_accesses20:25:111
264789998,1cyclictest32188-21sed20:25:253
264789997,1cyclictest31448-21acpi19:20:113
264789997,1cyclictest25895-21basename20:15:103
264789997,1cyclictest18832-21hostname20:00:133
264739996,2cyclictest3751-21cron19:30:002
264709997,1cyclictest8743-21cut20:45:171
264709997,1cyclictest19388-21grep21:05:281
264709992,6cyclictest6205-21diskstats20:40:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional