You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-06 - 07:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Sun Sep 06, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2611129469,9sleep20-21swapper/219:07:092
2632029367,8sleep00-21swapper/019:09:520
2606528370,9sleep30-21swapper/319:06:323
2627628164,13sleep10-21swapper/119:09:161
2641999290,28cyclictest0-21swapper/221:45:212
2642399280,24cyclictest0-21swapper/300:29:203
2641499230,22cyclictest0-21swapper/100:23:281
2641399221,15cyclictest0-21swapper/022:25:130
2641499200,18cyclictest0-21swapper/122:19:131
2641499190,18cyclictest0-21swapper/100:09:121
2641499190,18cyclictest0-21swapper/100:04:061
2641399196,6cyclictest31891-21ssh00:30:000
2641399195,8cyclictest11936-21grep23:39:590
26413991917,1cyclictest13-21ksoftirqd/000:30:180
26423991818,0cyclictest0-21swapper/322:41:363
26423991818,0cyclictest0-21swapper/322:41:363
26419991814,3cyclictest32-21ksoftirqd/222:55:122
2641499180,0cyclictest0-21swapper/123:44:081
2641399186,6cyclictest14329-21awk22:05:000
26413991813,5cyclictest13-21ksoftirqd/000:07:250
2641399180,18cyclictest0-21swapper/022:32:550
2642399170,1cyclictest0-21swapper/321:59:243
2641399175,6cyclictest3563-21awk21:20:000
26419991614,1cyclictest7628-21grep00:15:172
26419991612,3cyclictest3300-21cut23:15:172
2641999161,13cyclictest0-21swapper/200:13:082
2641499169,6cyclictest16154-21tr19:55:141
2641399164,3cyclictest225-21systemd-journal23:07:540
2641399160,2cyclictest0-21swapper/021:10:420
189092160,0chrt0-21swapper/222:26:102
2642399154,1cyclictest141rcu_preempt22:02:493
2641999153,11cyclictest2085-21ssh23:15:002
26419991511,3cyclictest11801-21uniq23:20:172
2641999151,13cyclictest0-21swapper/222:15:562
2641999150,1cyclictest0-21swapper/222:06:352
2641499153,1cyclictest0-21swapper/123:10:271
2641499152,9cyclictest16842-21cron21:05:011
26414991514,1cyclictest19219-21diskmemload21:40:191
26414991513,1cyclictest0-21swapper/123:15:161
2641499151,10cyclictest0-21swapper/121:38:251
2641499150,1cyclictest0-21swapper/122:00:451
2641399154,8cyclictest5427-21cups-browsed22:00:000
26413991513,1cyclictest13-21ksoftirqd/019:35:260
26413991512,2cyclictest13-21ksoftirqd/022:09:590
2642399143,5cyclictest0-21swapper/321:15:013
26423991411,2cyclictest4796-21ssh00:13:583
26423991410,3cyclictest5199-21grep23:35:193
26423991410,2cyclictest4548-21cron21:39:593
26423991410,2cyclictest18231-21wc22:45:213
26423991410,2cyclictest18049-21systemctl19:59:593
2642399140,14cyclictest0-21swapper/322:33:363
2641999142,11cyclictest0-21swapper/200:07:472
26419991411,2cyclictest15458-21munin-plugin-st22:25:002
26419991410,3cyclictest17577-21sed19:55:262
2641499145,5cyclictest5180-21apt-get23:55:131
2641499143,1cyclictest19219-21diskmemload22:08:161
26414991410,3cyclictest7327-21tr19:35:221
26414991410,2cyclictest8691-21tr00:35:171
2641399149,3cyclictest6462-21grep23:55:190
2641399144,5cyclictest225-21systemd-journal23:05:000
26413991410,3cyclictest29472-21grep20:20:250
26413991410,3cyclictest22798-21dump-pmu-power20:10:000
26413991410,2cyclictest9508-21grep22:20:270
2642399139,3cyclictest7898-21grep22:20:153
2642399139,3cyclictest28739-21basename23:49:593
2642399139,3cyclictest28565-21sed23:10:263
2642399139,3cyclictest16156-21cut00:20:183
2642399139,2cyclictest12959-21ssh23:40:143
2642399138,3cyclictest18264-21apache_accesses23:05:123
2642399131,6cyclictest225-21systemd-journal00:02:073
26423991310,2cyclictest32244-21cut22:15:173
26423991310,2cyclictest30733-21sed19:15:273
26423991310,2cyclictest15977-21cat22:05:193
26423991310,2cyclictest15432-21cron19:55:003
2642399130,1cyclictest0-21swapper/321:21:223
26419991310,2cyclictest23081-21basename20:10:152
26419991310,2cyclictest22825-21grep21:30:302
26419991310,2cyclictest1532-21cat22:35:212
26419991310,2cyclictest15105-21grep00:00:242
26419991310,2cyclictest13144-21basename23:59:592
2641999130,2cyclictest0-21swapper/223:36:222
2641499139,3cyclictest2252-21head22:55:181
2641499138,4cyclictest23232-21ssh22:10:151
2641499138,3cyclictest4974-21head19:30:211
2641499137,1cyclictest0-21swapper/121:19:151
2641499135,2cyclictest225-21systemd-journal19:49:591
2641499131,6cyclictest8655-21/usr/sbin/munin00:15:261
26414991310,2cyclictest29302-21grep23:50:141
26414991310,2cyclictest2243-21grep22:35:291
26414991310,2cyclictest21537-21grep21:30:191
26414991310,2cyclictest18898-21cron23:24:591
26414991310,2cyclictest11022-21ssh23:00:201
2641399139,3cyclictest9705-21apt20:50:000
2641399139,3cyclictest18623-21wc21:05:270
2641399139,2cyclictest1087-21grep20:30:180
2641399138,3cyclictest0-21swapper/022:57:340
2641399134,6cyclictest5417-21apache222:49:080
2641399131,7cyclictest2117-21perf23:15:000
26413991313,0cyclictest0-21swapper/000:23:010
26413991311,1cyclictest14067-21head19:50:180
26413991310,2cyclictest8717-21grep19:40:140
26413991310,2cyclictest6271-21chrt21:20:320
26413991310,2cyclictest4726-21cat21:40:000
26413991310,2cyclictest14769-21awk00:20:000
26413991310,2cyclictest14029-21cut21:25:230
2641399130,7cyclictest29705-21idleruntime-cro21:54:590
2641399130,12cyclictest31733-21wc20:25:280
2642399129,2cyclictest883-21sh22:55:013
2642399129,2cyclictest7426-21grep00:15:153
2642399129,2cyclictest6026-21tail23:55:163
2642399129,2cyclictest5468-21tr20:40:143
2642399129,2cyclictest4525-21grep23:15:273
2642399129,2cyclictest32407-21sort20:30:013
2642399129,2cyclictest28604-21sed20:20:173
2642399129,2cyclictest27919-21ssh23:30:123
2642399129,2cyclictest23921-21awk21:50:293
2642399129,2cyclictest21831-21ssh21:30:223
2642399129,2cyclictest19184-21chrt23:25:033
2642399129,2cyclictest11621-21grep20:50:293
2642399128,3cyclictest2827-21wc19:25:263
2642399128,3cyclictest22364-21sed20:05:253
2642399128,3cyclictest13078-21sed23:20:273
2642399128,2cyclictest13645-21basename21:45:143
2642399124,4cyclictest225-21systemd-journal20:19:593
2642399122,7cyclictest24023-21grep22:10:183
26423991210,1cyclictest0-21swapper/323:02:113
2642399120,11cyclictest16271-21tr22:25:143
2641999129,2cyclictest8371-21sed20:45:162
2641999129,2cyclictest32188-21head19:20:182
2641999129,2cyclictest30728-21sed19:15:272
2641999129,2cyclictest29509-21head21:15:212
2641999129,2cyclictest29274-21ssh23:30:202
2641999129,2cyclictest24582-21tr22:30:182
2641999129,2cyclictest14992-21grep21:00:152
2641999129,2cyclictest14373-21sed23:40:262
2641999128,3cyclictest23343-21grep21:50:242
2641999128,3cyclictest19906-21ssh21:10:152
2641999128,3cyclictest1817-21wc00:30:292
2641999120,12cyclictest0-21swapper/221:56:512
2641499129,2cyclictest32340-21sleep20:30:001
2641499129,2cyclictest32054-21cut19:20:171
2641499129,2cyclictest31540-21tail00:10:171
2641499129,2cyclictest31141-21sh19:20:011
2641499129,2cyclictest31096-21sed21:55:181
2641499129,2cyclictest24733-21sed20:10:261
2641499129,2cyclictest23299-21ls21:50:241
2641499129,2cyclictest1324-21tail00:30:241
2641499128,4cyclictest19606-21ssh21:29:271
2641499128,3cyclictest21020-21tr21:10:211
2641499128,2cyclictest20709-21ntp_kernel_err23:25:201
2641499127,4cyclictest14141-21taskset20:56:131
2641499127,4cyclictest14141-21taskset20:56:121
2641499126,2cyclictest225-21systemd-journal22:45:011
2641499126,2cyclictest225-21systemd-journal22:45:001
2641499120,1cyclictest0-21swapper/123:36:391
2641499120,1cyclictest0-21swapper/119:25:141
2641399129,2cyclictest9234-21tail00:35:190
2641399129,2cyclictest5912-21cron19:34:590
2641399129,2cyclictest5471-21tr21:40:150
2641399129,2cyclictest2743-21grep23:15:150
2641399129,2cyclictest26696-21basename19:10:140
2641399129,2cyclictest26194-21cut20:15:180
2641399129,2cyclictest15952-21sed21:00:220
2641399129,2cyclictest15116-21head00:00:230
2641399128,3cyclictest12553-21grep23:40:130
2641399128,2cyclictest30933-21wc23:50:260
26413991211,1cyclictest26404-21cyclictest21:47:300
26413991210,1cyclictest6876-21grep20:40:270
26413991210,1cyclictest21234-21grep21:30:180
2641399120,1cyclictest0-21swapper/022:53:390
2641399120,12cyclictest0-21swapper/022:38:440
2642399118,2cyclictest5363-21grep19:30:253
2642399118,2cyclictest4850-21ssh21:40:023
2642399118,2cyclictest445-21ssh23:51:453
2642399118,2cyclictest29942-21cat21:15:253
2642399118,2cyclictest26266-21chrt00:07:093
2642399118,2cyclictest2601-21ssh22:35:413
2642399118,2cyclictest17465-21ls21:05:173
2642399118,2cyclictest1304-21ssh22:55:093
2642399118,2cyclictest12535-21tr21:25:133
2642399118,2cyclictest11419-21chrt00:36:153
2642399118,2cyclictest11236-21tr19:45:163
2642399117,2cyclictest28913-21wc19:15:013
2642399113,4cyclictest225-21systemd-journal20:50:013
26423991110,1cyclictest55650irq/127-enp2s0-00:31:533
2641999118,2cyclictest8984-21cat22:40:152
2641999118,2cyclictest8984-21cat22:40:142
2641999118,2cyclictest7662-21timerandwakeup19:35:272
2641999118,2cyclictest7596-21ssh21:21:362
2641999118,2cyclictest7321-21dump-pmu-power20:45:012
2641999118,2cyclictest6284-21ssh21:40:202
2641999118,2cyclictest614-21cut20:30:142
2641999118,2cyclictest4566-21wc19:30:172
2641999118,2cyclictest396-21ssh23:51:432
2641999118,2cyclictest3806-21ssh21:39:252
2641999118,2cyclictest32316-21cron20:30:002
2641999118,2cyclictest31762-21ssh22:53:482
2641999118,2cyclictest25678-21cat00:25:272
2641999118,2cyclictest25559-21ssh23:47:332
2641999118,2cyclictest23509-21tr22:10:162
2641999118,2cyclictest22689-21rm23:07:212
2641999118,2cyclictest20890-21grep20:05:132
2641999118,2cyclictest19212-21cron21:10:002
2641999118,2cyclictest19107-21sed22:45:302
2641999118,2cyclictest18273-21sshd20:00:052
2641999118,2cyclictest16241-21ssh00:20:192
2641999118,2cyclictest14503-21rm21:25:272
2641999118,2cyclictest14376-21sh23:02:232
2641999118,2cyclictest10688-21sh19:44:592
2641999117,3cyclictest20256-21head23:25:172
2641999114,6cyclictest29955-21idleruntime-cro20:24:592
2641999111,1cyclictest0-21swapper/219:10:112
2641999110,10cyclictest6962-21expr22:00:162
2641499118,2cyclictest8297-21head20:45:161
2641499118,2cyclictest4939-21tail21:20:201
2641499118,2cyclictest461-21ssh23:32:511
2641499118,2cyclictest22462-21ssh22:28:571
2641499118,2cyclictest21235-21ssh21:49:551
2641499118,2cyclictest20013-21head20:00:251
2641499118,2cyclictest10721-21ssh22:21:191
2641499116,4cyclictest20972-21egrep23:45:121
2641499110,10cyclictest488-21cut20:30:141
2641499110,10cyclictest26520-21hwlatdetect22:50:251
2641499110,10cyclictest19230-21awk21:10:001
2641399118,2cyclictest4996-21cat20:40:000
2641399118,2cyclictest327-21strings22:15:190
2641399118,2cyclictest30769-21cut00:10:140
2641399118,2cyclictest29734-21grep19:15:180
2641399118,2cyclictest28432-21ssh23:30:150
2641399118,2cyclictest26674-21ssh22:11:360
2641399118,2cyclictest24754-21sed20:10:260
2641399118,2cyclictest23339-21chrt23:26:550
2641399118,2cyclictest18858-21df20:00:140
2641399118,2cyclictest16650-21grep19:55:170
2641399118,2cyclictest13443-21ssh23:20:450
2641399118,2cyclictest11182-21tr19:45:150
2641399110,10cyclictest11943-21taskset20:53:300
2642399109,1cyclictest0-21swapper/320:55:133
2642399109,1cyclictest0-21swapper/320:55:123
2642399108,1cyclictest8535-21chrt19:40:023
2642399108,1cyclictest651-21expr20:30:143
2642399108,1cyclictest2763-21apache_volume20:35:133
2642399108,1cyclictest23761-21grep20:10:183
2642399108,1cyclictest14972-21expr21:00:143
2642399107,2cyclictest6900-21grep19:35:183
2642399101,6cyclictest0-21swapper/319:20:143
2641999108,1cyclictest8116-21rm00:35:032
2641999108,1cyclictest3222-21expr20:35:162
2641999108,1cyclictest254072sleep220:15:082
2641999108,1cyclictest13174-21basename20:55:172
2641999108,1cyclictest13174-21basename20:55:162
2641999107,2cyclictest13342-21taskset19:50:152
2641499108,2cyclictest26509-21sh22:30:341
2641499108,1cyclictest3759-21cat20:35:201
2641499108,1cyclictest25821-21expr20:15:161
2641499108,1cyclictest14816-21expr19:50:271
2641499107,2cyclictest5518-21cat20:40:151
2641499107,2cyclictest28042-21sensors19:10:241
2641499107,2cyclictest23618-21ssh00:25:091
2641499107,2cyclictest20969-21cut20:05:141
2641499107,2cyclictest18617-21tr23:05:161
2641499106,3cyclictest28448-21grep20:20:161
2641499105,4cyclictest18388-21strings22:45:211
2641499100,2cyclictest225-21systemd-journal19:45:001
2641399108,1cyclictest239582sleep023:46:180
2641399107,2cyclictest14136-21chrt20:56:090
2641399107,2cyclictest14136-21chrt20:56:080
2641399107,2cyclictest12379-21ssh22:41:430
2641399107,2cyclictest12379-21ssh22:41:420
2641399106,3cyclictest32642-21grep19:20:230
2641399101,5cyclictest1387-21apt-get19:25:140
264239997,1cyclictest20168-21chrt20:01:553
264199997,1cyclictest2313-21kernelversion19:25:202
264199997,1cyclictest109132sleep219:45:022
264199997,1cyclictest10842-21cat20:50:212
264149998,1cyclictest10825-21cat20:50:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional