You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 05:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Jun 01, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
252921210,0sleep00-21swapper/023:45:300
633529279,9sleep00-21swapper/019:09:200
636028666,16sleep30-21swapper/319:09:383
628228564,6sleep20-21swapper/219:08:362
611228567,14sleep10-21swapper/119:06:271
30262590,0sleep10-21swapper/123:07:421
647299290,2cyclictest0-21swapper/021:41:490
647599280,25cyclictest0-21swapper/121:14:161
647599280,25cyclictest0-21swapper/121:14:151
647299270,1cyclictest0-21swapper/022:24:190
647299240,2cyclictest0-21swapper/022:09:270
648299230,0cyclictest0-21swapper/323:21:183
647899230,22cyclictest0-21swapper/200:33:272
647299230,19cyclictest0-21swapper/021:12:300
647299230,19cyclictest0-21swapper/021:12:290
648299220,1cyclictest0-21swapper/323:38:363
647899220,22cyclictest0-21swapper/222:41:502
647899220,22cyclictest0-21swapper/222:41:492
647899220,22cyclictest0-21swapper/222:19:032
647899220,21cyclictest0-21swapper/223:55:052
647599210,17cyclictest0-21swapper/122:55:141
647299210,3cyclictest0-21swapper/021:22:260
6482992017,1cyclictest55550irq/126-enp2s0-22:09:243
648299200,1cyclictest0-21swapper/321:57:013
647899200,2cyclictest0-21swapper/222:55:132
647299200,20cyclictest0-21swapper/023:15:330
647299200,1cyclictest0-21swapper/022:30:570
648299191,3cyclictest141rcu_preempt21:19:003
647899191,17cyclictest31688-21diskmemload23:30:122
647899190,18cyclictest0-21swapper/223:17:062
647899190,16cyclictest0-21swapper/221:29:272
647599190,2cyclictest0-21swapper/100:28:281
647599190,1cyclictest0-21swapper/123:02:471
6472991918,1cyclictest0-21swapper/023:42:200
6472991918,1cyclictest0-21swapper/000:35:140
648299180,1cyclictest0-21swapper/323:03:123
6478991818,0cyclictest0-21swapper/222:33:202
647899180,1cyclictest0-21swapper/221:17:082
647599180,17cyclictest0-21swapper/121:53:401
647299180,0cyclictest0-21swapper/022:54:170
6472991716,1cyclictest13-21ksoftirqd/022:11:320
647299171,13cyclictest31688-21diskmemload00:03:390
647299170,4cyclictest0-21swapper/021:59:560
647299170,1cyclictest0-21swapper/021:35:410
647299170,16cyclictest0-21swapper/023:04:340
648299162,1cyclictest31688-21diskmemload23:27:163
648299160,2cyclictest0-21swapper/323:54:583
647899160,4cyclictest0-21swapper/223:50:162
647899160,3cyclictest0-21swapper/222:26:212
647599161,15cyclictest0-21swapper/123:52:471
6475991611,3cyclictest0-21swapper/122:31:561
6475991610,2cyclictest0-21swapper/121:45:051
6472991615,1cyclictest0-21swapper/023:35:130
6472991613,2cyclictest13-21ksoftirqd/019:35:210
647299160,2cyclictest0-21swapper/023:07:260
647299160,2cyclictest0-21swapper/021:31:340
647299160,0cyclictest0-21swapper/021:50:090
648299153,7cyclictest11709-21cpuspeed_turbos22:15:143
648299150,1cyclictest0-21swapper/322:35:463
648299150,0cyclictest0-21swapper/323:48:393
647899152,4cyclictest0-21swapper/221:45:132
647899150,2cyclictest0-21swapper/221:37:412
647599158,6cyclictest19573-21cron20:45:001
647599157,4cyclictest25-21ksoftirqd/100:08:501
647599154,7cyclictest8737-21munin-node23:15:181
647299150,1cyclictest0-21swapper/000:28:050
97842140,1sleep10-21swapper/100:30:121
648299142,5cyclictest8977-21expr00:10:103
648299140,1cyclictest0-21swapper/322:32:193
648299140,14cyclictest0-21swapper/322:48:143
648299140,12cyclictest9998-21cron20:25:003
647899149,2cyclictest14336-21cron23:15:002
647899145,9cyclictest0-21swapper/200:16:132
647899141,13cyclictest0-21swapper/200:02:572
647899140,1cyclictest0-21swapper/223:40:182
647899140,12cyclictest0-21swapper/223:06:392
647599142,1cyclictest0-21swapper/122:46:161
647599141,12cyclictest31688-21diskmemload22:17:161
647599140,2cyclictest0-21swapper/123:14:391
647599140,1cyclictest0-21swapper/123:34:591
647299149,3cyclictest10194-21wc20:25:000
6472991411,2cyclictest6919-21wc20:15:260
6472991410,3cyclictest14362-21head20:30:260
6472991410,2cyclictest32274-21wc20:00:290
6472991410,2cyclictest25844-21cut19:50:160
647299140,6cyclictest0-21swapper/023:55:180
647299140,1cyclictest0-21swapper/023:25:520
648299139,3cyclictest26694-21cron21:00:013
648299134,3cyclictest20503-21diskstats22:20:163
648299132,4cyclictest0-21swapper/321:25:233
648299131,1cyclictest31688-21diskmemload21:22:033
6482991310,2cyclictest30630-21ssh22:25:453
6482991310,2cyclictest29307-21cron21:05:003
6482991310,2cyclictest25081-21sed19:45:273
6482991310,2cyclictest15497-21cat22:55:253
648299130,1cyclictest31688-21diskmemload00:16:023
648299130,1cyclictest141rcu_preempt00:07:553
648299130,1cyclictest0-21swapper/322:02:343
647899139,3cyclictest23703-21ls20:50:262
647899139,3cyclictest17601-21cat23:35:252
647899139,3cyclictest11036-21sed20:25:182
647899133,1cyclictest225-21systemd-journal00:07:572
647899131,9cyclictest21980-21runrttasks22:20:322
6478991310,2cyclictest5632-21grep20:15:142
6478991310,2cyclictest29462-21cat21:05:002
6478991310,2cyclictest13630-21cat20:30:192
647899130,13cyclictest0-21swapper/223:28:342
647899130,13cyclictest0-21swapper/221:50:172
647899130,11cyclictest744-21apt-get20:05:162
647599139,3cyclictest23525-21tr23:20:141
647599139,3cyclictest17916-21sed19:30:261
647599139,3cyclictest17804-21grep23:35:261
647599138,2cyclictest0-21swapper/121:05:151
6475991311,2cyclictest0-21swapper/123:25:521
6475991311,1cyclictest25-21ksoftirqd/100:14:471
6475991310,2cyclictest28874-21grep22:05:271
6475991310,2cyclictest26744-21sed00:00:181
6475991310,2cyclictest20381-21cron22:40:011
6475991310,2cyclictest18617-21cat23:55:201
647599130,12cyclictest28890-21head21:00:261
647599130,12cyclictest26803-21ssh21:25:261
647299139,3cyclictest29656-21wc22:25:230
647299139,2cyclictest25791-21sed21:25:160
647299135,7cyclictest17202-21sendmail20:40:000
6472991310,2cyclictest9634-21rm23:30:330
6472991310,2cyclictest30477-21grep22:45:200
6472991310,2cyclictest23636-21wc20:50:270
647299130,11cyclictest0-21swapper/000:07:130
35842130,0chrt0-21swapper/122:29:581
648299129,2cyclictest5965-21uniq22:50:183
648299129,2cyclictest3723-21uniq20:10:193
648299129,2cyclictest30471-21cat19:59:593
648299129,2cyclictest2118-21chrt00:25:173
648299129,2cyclictest17001-21grep19:30:173
648299129,2cyclictest15897-21uname23:15:183
648299128,3cyclictest10456-21ssh21:35:203
648299124,6cyclictest18216-21tune2fs23:55:173
648299122,1cyclictest0-21swapper/300:20:093
648299122,1cyclictest0-21swapper/300:20:083
6482991212,0cyclictest0-21swapper/323:05:143
648299120,9cyclictest13834-21pluginstate20:30:213
648299120,1cyclictest0-21swapper/323:12:443
647899129,2cyclictest9195-21tr19:15:152
647899129,2cyclictest676-21grep23:45:142
647899129,2cyclictest3849-21tail20:10:202
647899129,2cyclictest31883-21ssh00:23:542
647899129,2cyclictest31883-21ssh00:23:532
647899129,2cyclictest24659-21grep19:45:232
647899129,2cyclictest21517-21grep20:45:272
647899129,2cyclictest19082-21grep20:40:262
647899129,2cyclictest1102-21ssh21:30:142
6478991210,1cyclictest31678-21cron21:10:002
647899120,1cyclictest0-21swapper/200:29:302
647599129,2cyclictest866-21ssh21:30:121
647599129,2cyclictest21843-21cut19:40:161
647599129,2cyclictest21498-21tail20:45:271
647599129,2cyclictest21245-21sed22:20:211
647599129,2cyclictest15825-21cut20:35:171
647599129,2cyclictest15241-21fgrep19:25:241
647599129,2cyclictest1236-21df20:05:181
647599128,3cyclictest28824-21cut19:55:151
6475991210,1cyclictest25-21ksoftirqd/121:40:261
647299129,2cyclictest4445-21wc20:10:240
647299129,2cyclictest28821-21sed21:00:260
647299129,2cyclictest24158-21grep19:45:170
647299129,2cyclictest18669-21wc00:15:250
647299129,2cyclictest18407-21munin-plugin-st19:35:000
647299128,3cyclictest881-21sh00:25:000
647299128,3cyclictest881-21sh00:25:000
647299128,3cyclictest20064-21grep22:00:240
647299126,2cyclictest225-21systemd-journal23:55:000
647299123,7cyclictest14574-21cups-browsed20:25:240
648299118,2cyclictest8267-21grep23:30:163
648299118,2cyclictest32263-21grep21:10:133
648299118,2cyclictest32263-21grep21:10:133
648299118,2cyclictest31220-21sed21:05:283
648299118,2cyclictest2917-21ssh22:10:073
648299118,2cyclictest27208-21uname00:00:263
648299118,2cyclictest26904-21cat19:50:213
648299118,2cyclictest26283-21seq21:45:153
648299118,2cyclictest2558-21grep21:50:153
648299118,2cyclictest2442-21grep20:05:303
648299118,2cyclictest24260-21rm23:40:053
648299118,2cyclictest22527-21seq20:50:153
648299118,2cyclictest22126-21ssh22:40:193
648299118,2cyclictest22126-21ssh22:40:183
648299118,2cyclictest21799-21chrt20:48:113
648299118,2cyclictest20159-21rm00:35:273
648299118,2cyclictest12570-21needreboot19:20:213
648299118,2cyclictest1100-21sort21:30:143
648299118,2cyclictest10674-21which00:30:173
648299115,2cyclictest225-21systemd-journal20:35:353
648299110,1cyclictest0-21swapper/321:41:343
647899119,1cyclictest10462-21grep21:55:142
647899119,1cyclictest10117-21fgrep00:10:172
647899119,1cyclictest0-21swapper/200:36:172
647899118,2cyclictest8154-21tr20:20:162
647899118,2cyclictest32579-21expr21:10:152
647899118,2cyclictest32579-21expr21:10:152
647899118,2cyclictest28991-21ssh22:45:012
647899118,2cyclictest28032-21ssh22:05:202
647899118,2cyclictest27632-21ssh23:03:062
647899118,2cyclictest26060-21grep19:50:162
647899118,2cyclictest24340-21ssh21:44:092
647899118,2cyclictest23040-21ssh23:20:032
647899118,2cyclictest22431-21basename19:40:222
647899118,2cyclictest20683-21ssh22:00:312
647899118,2cyclictest14654-21grep19:25:182
647899118,2cyclictest12773-21grep22:35:162
647899116,4cyclictest13050-21awk22:55:002
647899114,6cyclictest20773-21idleruntime-cro19:40:012
647899111,9cyclictest0-21swapper/221:24:102
647899110,10cyclictest3807-21uniq22:10:182
647899110,10cyclictest17066-21gltestperf19:30:162
647599119,1cyclictest23691-21tr19:45:161
647599118,2cyclictest7663-21wc20:19:591
647599118,2cyclictest5940-21ssh22:11:071
647599118,2cyclictest3859-21grep20:10:191
647599118,2cyclictest32454-21grep20:00:301
647599118,2cyclictest28921-21ssh00:21:351
647599118,2cyclictest28921-21ssh00:21:341
647599118,2cyclictest26575-21chrt23:40:321
647599118,2cyclictest23724-21wc20:50:261
647599118,2cyclictest23236-21ssh21:24:041
647599118,2cyclictest18605-21ssh22:00:151
647599118,2cyclictest13264-21ssh21:56:161
647599118,2cyclictest12136-21taskset20:27:001
647599118,2cyclictest11096-21tail21:35:251
647599118,2cyclictest10948-21taskset19:16:571
647599117,3cyclictest18936-21sh00:35:181
647599117,3cyclictest13389-21tr20:30:161
647599117,2cyclictest0-21swapper/119:50:151
647599113,4cyclictest225-21systemd-journal23:46:351
647599110,3cyclictest0-21swapper/122:41:121
647599110,3cyclictest0-21swapper/122:41:121
647299118,2cyclictest8695-21ssh21:15:190
647299118,2cyclictest6229-21ssh23:10:010
647299118,2cyclictest30639-21wc21:05:210
647299118,2cyclictest28616-21rm21:45:580
647299118,2cyclictest26439-21taskset20:56:450
647299118,2cyclictest25494-21ssh22:42:280
647299118,2cyclictest25494-21ssh22:42:280
647299118,2cyclictest18816-21ssh22:19:370
647299118,2cyclictest15575-21sed19:25:290
647299118,2cyclictest14438-21sed22:35:270
647299118,2cyclictest13773-21ssh00:12:480
647299118,2cyclictest10696-21sed19:15:270
647299114,3cyclictest0-21swapper/023:20:130
647299112,6cyclictest31688-21diskmemload22:57:000
647299110,2cyclictest0-21swapper/020:05:160
648299108,1cyclictest9482-21expr19:15:163
648299108,1cyclictest7171-21expr19:10:133
648299108,1cyclictest7171-21expr19:10:123
648299108,1cyclictest21029-21cat19:40:113
648299108,1cyclictest12075-21taskset20:26:143
648299107,2cyclictest5682-21grep20:15:143
648299107,2cyclictest18953-21grep19:35:133
648299107,2cyclictest15831-21taskset19:27:393
647899108,1cyclictest8036-21systemctl19:10:202
647899108,1cyclictest8036-21systemctl19:10:192
647899108,1cyclictest29691-21ls19:55:242
647899108,1cyclictest15189-21aten2_r7power_v20:35:142
647899107,2cyclictest25219-21grep20:55:172
647899107,2cyclictest12900-21cat19:20:272
647599108,1cyclictest86782sleep119:13:251
647599108,1cyclictest86782sleep119:13:241
647599108,1cyclictest5067-21ssh22:50:071
647599108,1cyclictest12063-21diskstats19:20:171
647599107,2cyclictest25340-21grep20:55:161
647599107,2cyclictest18700-21cut19:35:131
647599107,2cyclictest17767-21ssh00:15:181
647599106,3cyclictest7935-21grep20:20:151
647599100,9cyclictest7856-21ssh21:15:021
647299107,2cyclictest6955-21grep19:10:130
647299107,2cyclictest6955-21grep19:10:120
647299107,2cyclictest28952-21head19:55:160
647299107,2cyclictest23050-21taskset19:43:380
647299107,2cyclictest21328-21wc20:45:260
647299107,2cyclictest19398-21chrt20:42:460
647299107,2cyclictest13487-21sh00:31:450
64829997,1cyclictest30981-21egrep20:00:163
64829997,1cyclictest18450-21cat20:40:193
64789990,9cyclictest326482sleep220:01:542
64729996,2cyclictest13133-21grep19:20:270
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional