You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-20 - 23:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Sun Sep 20, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1820329175,11sleep00-21swapper/007:05:340
1835728371,8sleep20-21swapper/207:07:252
1841628269,9sleep30-21swapper/307:08:103
1832627868,6sleep10-21swapper/107:06:591
157962560,0sleep30-21swapper/312:05:493
133872480,0sleep10-21swapper/111:06:481
1865199361,2cyclictest5706-21grep08:55:261
1865199310,30cyclictest0-21swapper/109:26:411
1865499280,28cyclictest0-21swapper/212:12:352
1865499270,27cyclictest0-21swapper/212:23:362
18649992727,0cyclictest0-21swapper/011:38:310
1865499260,25cyclictest0-21swapper/211:05:182
1865199261,1cyclictest31965-21apt-get08:45:141
18656992523,1cyclictest0-21swapper/312:10:173
1865499250,24cyclictest0-21swapper/211:55:002
18654992422,1cyclictest0-21swapper/209:48:342
1865199247,1cyclictest141rcu_preempt10:02:401
1865199240,1cyclictest0-21swapper/111:17:441
18654992319,1cyclictest32-21ksoftirqd/211:55:282
18651992320,3cyclictest25-21ksoftirqd/112:15:451
1865199230,23cyclictest0-21swapper/109:55:241
1865699228,6cyclictest0-21swapper/312:20:133
1865499220,22cyclictest0-21swapper/211:31:282
1865499220,22cyclictest0-21swapper/211:04:282
1865499220,22cyclictest0-21swapper/210:21:172
1865499220,22cyclictest0-21swapper/210:16:342
1865499220,21cyclictest0-21swapper/209:54:242
1865499220,1cyclictest0-21swapper/209:58:572
1865199220,1cyclictest0-21swapper/112:24:011
18656992118,1cyclictest0-21swapper/310:50:133
1865499210,21cyclictest0-21swapper/209:21:482
1865499210,21cyclictest0-21swapper/209:21:472
1865499210,20cyclictest0-21swapper/209:33:142
1865499210,1cyclictest141rcu_preempt11:29:332
1865499210,1cyclictest0-21swapper/207:10:132
1865499210,19cyclictest0-21swapper/209:25:242
18656992018,1cyclictest11268-21diskmemload10:41:523
18656992018,1cyclictest11268-21diskmemload10:41:523
18656992015,2cyclictest0-21swapper/311:30:163
18656992013,6cyclictest3309-21sensors_temp08:50:253
1865699200,18cyclictest0-21swapper/311:41:563
18654992019,1cyclictest0-21swapper/210:06:242
1865499200,20cyclictest0-21swapper/210:14:192
1865199200,17cyclictest0-21swapper/111:12:331
1865199200,0cyclictest0-21swapper/111:45:071
1865199200,0cyclictest0-21swapper/111:45:061
18649992013,1cyclictest474-21idleruntime-cro07:40:010
1864999200,2cyclictest0-21swapper/010:22:070
1865499191,14cyclictest0-21swapper/210:59:122
1865199192,7cyclictest9079-21latency_hist09:05:001
1865199191,9cyclictest28536-21gltestperf11:35:171
1865199190,1cyclictest0-21swapper/109:18:591
18649991918,1cyclictest0-21swapper/012:12:350
18656991816,1cyclictest0-21swapper/309:55:123
18654991810,1cyclictest0-21swapper/209:40:582
1865499180,18cyclictest0-21swapper/210:27:442
1865499180,15cyclictest0-21swapper/209:12:452
1865199189,2cyclictest15853-21grep10:10:171
18649991818,0cyclictest0-21swapper/010:38:400
18649991817,1cyclictest0-21swapper/012:23:360
18649991816,2cyclictest0-21swapper/010:31:050
18649991812,3cyclictest13-21ksoftirqd/012:35:130
1864999180,1cyclictest0-21swapper/012:30:030
1865699175,11cyclictest14918-21/usr/sbin/munin10:10:273
18656991716,1cyclictest11268-21diskmemload11:00:583
18656991716,1cyclictest0-21swapper/311:15:413
18656991715,1cyclictest29618-21ssh11:55:163
18656991713,3cyclictest3838-21grep07:45:183
18656991711,4cyclictest0-21swapper/311:27:243
18654991712,1cyclictest141rcu_preempt11:22:472
1865499171,0cyclictest141rcu_preempt09:35:042
1865499170,14cyclictest0-21swapper/212:26:492
1865499170,0cyclictest0-21swapper/212:34:202
18651991715,1cyclictest0-21swapper/109:45:211
18651991714,2cyclictest25-21ksoftirqd/111:44:581
1865199170,9cyclictest0-21swapper/109:30:131
1865199170,16cyclictest0-21swapper/110:30:451
1864999170,16cyclictest0-21swapper/009:30:550
183632170,0sleep30-21swapper/310:30:483
1865699168,8cyclictest11268-21diskmemload10:49:383
1865699168,8cyclictest0-21swapper/309:26:183
18656991613,2cyclictest0-21swapper/312:19:233
18656991612,3cyclictest0-21swapper/311:10:163
18654991615,1cyclictest0-21swapper/211:40:132
1865499160,16cyclictest0-21swapper/209:18:212
1865199163,11cyclictest38150irq/131-snd_hda11:50:161
1865199162,13cyclictest14217-21ssh10:48:231
18649991615,1cyclictest0-21swapper/011:05:180
18649991614,1cyclictest0-21swapper/009:13:200
1864999161,15cyclictest0-21swapper/011:16:550
1865699159,2cyclictest19007-21ssh11:49:183
1865699159,2cyclictest19007-21ssh11:49:183
1865699153,9cyclictest0-21swapper/311:50:343
1865699153,10cyclictest0-21swapper/312:04:053
18656991514,1cyclictest0-21swapper/310:38:163
18656991513,1cyclictest11268-21diskmemload09:15:273
18656991511,3cyclictest0-21swapper/310:20:123
1865699151,12cyclictest3356-21ssh09:43:463
18656991511,2cyclictest11268-21diskmemload10:06:123
18654991513,1cyclictest32-21ksoftirqd/210:33:242
1865499150,15cyclictest0-21swapper/212:36:012
1865499150,15cyclictest0-21swapper/211:48:492
1865499150,15cyclictest0-21swapper/211:48:492
1865499150,14cyclictest0-21swapper/212:01:202
1865499150,14cyclictest0-21swapper/210:04:062
1865499150,13cyclictest0-21swapper/207:15:142
1865199155,5cyclictest4907-21apt-get12:00:131
18651991513,1cyclictest38150irq/131-snd_hda11:34:241
18651991511,3cyclictest38150irq/131-snd_hda12:27:371
18649991513,1cyclictest0-21swapper/011:01:350
1865699149,5cyclictest19419-21vmstat08:15:273
1865699143,10cyclictest29920-21ssh10:19:053
18656991413,1cyclictest0-21swapper/312:25:183
18656991412,1cyclictest11268-21diskmemload09:21:393
18656991412,1cyclictest11268-21diskmemload09:21:393
18656991412,1cyclictest0-21swapper/307:50:253
18656991411,2cyclictest12808-21wget09:30:143
18656991410,1cyclictest9281-21ssh10:25:263
1865499149,1cyclictest141rcu_preempt10:45:202
1865499145,5cyclictest0-21swapper/208:05:132
1865499140,13cyclictest0-21swapper/210:44:252
1865499140,13cyclictest0-21swapper/210:44:242
1865199149,3cyclictest8314-21ssh10:25:181
1865199141,10cyclictest29428-21latency_hist08:40:011
18649991413,1cyclictest0-21swapper/010:25:290
1864999140,2cyclictest0-21swapper/009:52:460
1865699139,3cyclictest6456-21sh11:22:003
1865699139,2cyclictest0-21swapper/311:35:073
1865699139,0cyclictest0-21swapper/309:50:133
1865699137,3cyclictest0-21swapper/311:05:133
18656991312,1cyclictest11268-21diskmemload12:33:193
18656991310,2cyclictest31759-21tune2fs12:35:183
18656991310,2cyclictest29051-21munin-plugin-st09:40:013
18656991310,2cyclictest22190-21ls08:25:003
18656991310,2cyclictest13632-21grep09:10:253
1865499130,13cyclictest0-21swapper/211:38:352
1865499130,13cyclictest0-21swapper/211:19:172
1865499130,12cyclictest26960-21wc08:34:592
1865499130,12cyclictest0-21swapper/210:36:462
1865199139,3cyclictest21971-21tail07:15:171
1865199134,5cyclictest1330-21ssh11:00:141
1865199131,2cyclictest141rcu_preempt09:22:251
1865199131,2cyclictest141rcu_preempt09:22:241
18651991310,3cyclictest7135-21apache208:11:251
18651991310,2cyclictest1951-21grep08:50:141
18651991310,2cyclictest18615-21wc10:50:261
1865199130,12cyclictest31726-21wc08:44:591
1865199130,12cyclictest0-21swapper/112:35:011
1864999139,3cyclictest18345-21sort08:15:170
1864999136,1cyclictest141rcu_preempt12:19:250
18649991312,1cyclictest0-21swapper/007:10:130
1864999131,12cyclictest0-21swapper/010:45:190
18649991310,2cyclictest3248-21tr11:20:150
18649991310,2cyclictest30799-21grep09:40:210
18649991310,2cyclictest24661-21latency_hist10:55:010
1864999130,3cyclictest0-21swapper/010:13:560
1864999130,13cyclictest0-21swapper/011:48:580
1864999130,13cyclictest0-21swapper/011:48:570
1865699129,2cyclictest8711-21grep07:55:183
1865699127,4cyclictest12394-21runrttasks08:01:503
1865699123,9cyclictest0-21swapper/310:56:053
1865699122,9cyclictest30644-21acpi10:00:123
18656991210,1cyclictest6013-21tr09:45:163
1865499129,2cyclictest3731-21sort07:45:172
1865499129,2cyclictest3146-21head08:50:222
1865499129,2cyclictest31382-21cut07:35:182
1865499129,2cyclictest29848-21tr08:40:162
1865499129,2cyclictest19815-21awk08:19:592
1865499129,2cyclictest16018-21tail08:10:192
1865499129,2cyclictest15236-21grep12:05:262
1865499129,2cyclictest11299-21cut08:00:172
1865499124,3cyclictest516-21gdbus08:00:012
1865499121,4cyclictest225-21systemd-journal08:20:252
18654991210,1cyclictest7684-21sleep07:55:002
18654991210,1cyclictest28817-21grep08:35:272
18654991210,1cyclictest26804-21sed07:25:172
1865499120,1cyclictest0-21swapper/210:53:222
1865199129,2cyclictest7502-21cut10:05:161
1865199129,2cyclictest29566-21sed11:55:141
1865199129,2cyclictest2746-21cut11:20:131
1865199129,2cyclictest24977-21cut10:35:181
1865199129,2cyclictest13578-21grep12:05:141
1865199129,2cyclictest12546-21cut09:10:161
1865199129,2cyclictest11179-21tr11:25:141
1865199128,3cyclictest13688-21cut08:05:171
1865199120,0cyclictest0-21swapper/107:28:041
1864999129,2cyclictest8922-21cat09:05:010
1864999129,2cyclictest26958-21cat08:34:590
1864999129,2cyclictest16016-21grep08:10:190
1864999129,2cyclictest14091-21grep12:05:170
1864999129,2cyclictest13693-21grep08:05:170
1864999129,2cyclictest10726-21ssh09:47:480
1864999124,4cyclictest22680-21/usr/sbin/munin09:35:230
1864999124,4cyclictest0-21swapper/011:31:030
1864999122,1cyclictest141rcu_preempt09:25:010
1864999122,1cyclictest141rcu_preempt09:25:000
1864999121,6cyclictest25778-21munin-run07:25:010
18649991211,1cyclictest0-21swapper/011:58:070
18649991211,1cyclictest0-21swapper/009:58:570
18649991210,1cyclictest5894-21grep08:55:260
18649991210,1cyclictest5609-21sed11:40:260
1864999120,11cyclictest27166-21tr07:25:200
1865699118,2cyclictest5902-21grep08:55:273
1865699118,2cyclictest28307-21cat07:30:013
1865699118,2cyclictest20509-21grep07:10:263
1865699118,2cyclictest1124-21expr07:40:133
1865699114,3cyclictest225-21systemd-journal09:05:003
1865699114,3cyclictest225-21systemd-journal07:20:013
1865699111,9cyclictest14684-21taskset08:05:453
1865499119,2cyclictest7135-21apache211:10:172
1865499118,2cyclictest30786-21head12:15:172
1865499118,2cyclictest29057-21cut07:30:172
1865499118,2cyclictest29057-21cut07:30:162
1865499118,2cyclictest1604-21chrt08:49:342
1865499110,10cyclictest24425-21cut07:20:172
1865199119,1cyclictest25091-21tr08:30:171
1865199118,2cyclictest991-21tr07:40:131
1865199118,2cyclictest7442-21taskset07:52:141
1865199118,2cyclictest4952-21pluginstate09:45:001
1865199118,2cyclictest31054-21tr07:35:161
1865199118,2cyclictest28347-21ssh10:17:521
1865199118,2cyclictest27574-21sh10:55:451
1865199118,2cyclictest25364-21head07:20:271
1865199118,2cyclictest23155-21wc12:10:251
1865199118,2cyclictest21074-21ssh09:35:071
1865199118,2cyclictest14840-21ssh09:50:171
1865199118,2cyclictest11304-21ls09:10:011
1865199118,2cyclictest10197-21cat08:00:001
1865199117,3cyclictest31663-21cut10:20:131
1865199117,3cyclictest11322-21head08:00:181
1865199117,2cyclictest24729-21grep12:30:281
1864999119,1cyclictest4345-21ls07:45:220
1864999119,1cyclictest0-21swapper/009:25:240
1864999118,2cyclictest6169-21cut12:00:200
1864999118,2cyclictest32614-21sed10:00:250
1864999118,2cyclictest28818-21sed08:35:270
1864999118,2cyclictest23863-21cat08:25:260
1864999118,2cyclictest20977-21tr11:50:160
1864999118,2cyclictest18709-21sh11:30:010
1864999118,1cyclictest0-21swapper/010:42:280
1864999118,1cyclictest0-21swapper/010:42:270
18649991110,1cyclictest0-21swapper/012:27:500
1864999110,11cyclictest0-21swapper/010:07:250
1864999110,10cyclictest23136-21taskset07:17:070
1865699109,1cyclictest0-21swapper/308:25:183
1865699107,2cyclictest9416-21grep09:05:153
1865699107,2cyclictest32749-21hddtemp_smartct08:45:173
1865699107,2cyclictest15248-21chrt08:10:043
1865699106,3cyclictest26455-21unixbench_multi08:30:303
1865699105,2cyclictest29545-21taskset08:40:023
1865699105,1cyclictest273-21systemd-udevd07:34:033
1865699105,1cyclictest273-21systemd-udevd07:34:033
1865699103,3cyclictest29088-21sshd08:37:243
1865699100,10cyclictest31045-21expr07:35:153
1865699100,10cyclictest24547-21cat07:20:183
1865499108,1cyclictest85352sleep209:00:352
1865499108,1cyclictest22566-21cut08:25:152
1865499107,2cyclictest9706-21grep09:05:162
1865199108,1cyclictest32614-21cut10:40:151
1865199108,1cyclictest32614-21cut10:40:141
1865199108,1cyclictest20165-21cut08:20:131
1865199107,2cyclictest4814-21grep07:45:271
1865199107,2cyclictest29859-21sort07:30:261
1865199107,2cyclictest29859-21sort07:30:251
1865199107,2cyclictest18775-21grep08:15:201
1864999108,2cyclictest25676-21ssh10:15:460
1864999108,2cyclictest21075-21sh09:15:180
1864999108,1cyclictest5602-21cut07:50:130
1864999108,1cyclictest32412-21expr08:45:150
1864999108,1cyclictest30298-21tail08:40:180
1864999108,1cyclictest24959-21ssh10:55:050
1864999108,1cyclictest21171-21grep08:20:190
1864999108,1cyclictest1961-21basename08:50:150
1864999108,1cyclictest18816-21grep11:10:190
1864999108,1cyclictest11488-21cut09:10:000
1864999108,1cyclictest1049-21cat07:40:130
1864999107,2cyclictest29799-21sshd07:30:250
1864999107,2cyclictest29799-21sshd07:30:240
186549997,1cyclictest61992sleep208:57:392
186549996,2cyclictest979-21tr07:40:132
186519997,1cyclictest19908-21wc07:10:191
186519996,2cyclictest22469-21aten2_r7power_c08:25:151
186499997,2cyclictest10400-21cat08:00:020
186499997,1cyclictest9244-21gzip07:55:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional