You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 01:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Wed Apr 22, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2056729573,6sleep10-21swapper/107:07:501
2070828875,9sleep00-21swapper/007:09:390
2044028774,9sleep30-21swapper/307:06:123
2050628269,9sleep20-21swapper/207:07:032
276782490,0sleep30-21swapper/311:53:353
173242430,0sleep00-21swapper/009:30:350
20824992422,1cyclictest22291-21apt-get08:20:131
2082499240,22cyclictest0-21swapper/109:49:431
2082999230,0cyclictest0-21swapper/310:24:233
2082999230,0cyclictest0-21swapper/309:17:073
2082999221,3cyclictest0-21swapper/312:16:343
2082999220,1cyclictest0-21swapper/309:23:263
2082999220,0cyclictest0-21swapper/310:41:323
20826992220,2cyclictest32-21ksoftirqd/209:49:412
2082499220,1cyclictest0-21swapper/109:30:031
20829992119,2cyclictest26451-21kworker/3:112:20:423
20829992116,4cyclictest26451-21kworker/3:112:05:223
2082999200,2cyclictest0-21swapper/312:27:383
2082499200,20cyclictest0-21swapper/112:18:221
2082499200,20cyclictest0-21swapper/109:54:221
276882190,1sleep10-21swapper/109:17:491
2082999191,8cyclictest0-21swapper/309:25:143
20829991917,1cyclictest0-21swapper/311:11:073
2082499190,17cyclictest0-21swapper/111:19:331
2082499190,17cyclictest0-21swapper/110:33:431
2082999180,1cyclictest0-21swapper/312:31:463
2082999180,1cyclictest0-21swapper/309:55:583
2082999180,14cyclictest0-21swapper/310:19:033
2082699180,0cyclictest0-21swapper/209:24:252
20824991817,1cyclictest0-21swapper/108:55:031
20824991812,2cyclictest225-21systemd-journal07:44:591
2082499180,4cyclictest0-21swapper/109:55:141
2082499180,17cyclictest0-21swapper/109:42:361
2082999174,1cyclictest141rcu_preempt12:11:203
20829991717,0cyclictest0-21swapper/311:07:203
20829991716,1cyclictest0-21swapper/311:16:533
2082999170,2cyclictest0-21swapper/311:58:183
2082999170,1cyclictest0-21swapper/311:48:383
2082999170,14cyclictest0-21swapper/309:54:523
2082699177,8cyclictest26229-21ssh12:11:492
2082699175,7cyclictest11184-21/usr/sbin/munin10:25:262
2082699171,15cyclictest5668-21chrt10:03:022
20824991712,2cyclictest225-21systemd-journal10:48:401
2082099171,5cyclictest225-21systemd-journal10:20:010
2082999161,7cyclictest21119-21kworker/u8:011:04:103
2082999160,2cyclictest0-21swapper/310:27:273
2082999160,16cyclictest0-21swapper/310:39:113
2082999160,0cyclictest0-21swapper/311:28:273
2082999160,0cyclictest0-21swapper/309:47:443
20826991612,3cyclictest5084-21cron07:44:592
2082699161,12cyclictest0-21swapper/210:58:292
2082699160,1cyclictest55550irq/126-enp2s0-12:24:142
20824991614,1cyclictest0-21swapper/112:02:371
2082499161,1cyclictest0-21swapper/111:41:571
2082499160,16cyclictest0-21swapper/110:06:401
2082099166,3cyclictest18724-21apt-get10:50:130
20820991615,1cyclictest0-21swapper/010:42:210
2082099160,1cyclictest0-21swapper/009:18:070
2082099160,0cyclictest0-21swapper/009:42:310
20829991512,3cyclictest21119-21kworker/u8:010:53:533
2082999150,1cyclictest0-21swapper/312:35:223
2082999150,0cyclictest0-21swapper/309:38:433
2082699155,3cyclictest13885-21diskmemload09:57:142
2082699153,11cyclictest15687-21apt-get08:05:162
2082699150,2cyclictest0-21swapper/210:37:012
20824991511,3cyclictest31856-21wc08:40:011
20824991510,3cyclictest14655-21cron09:29:591
2082499150,14cyclictest0-21swapper/110:55:441
2082099157,7cyclictest6519-21munin-run08:55:010
2082099150,3cyclictest0-21swapper/009:49:390
2082999149,5cyclictest26451-21kworker/3:111:35:133
20829991414,0cyclictest0-21swapper/310:12:193
20829991412,1cyclictest0-21swapper/312:01:353
20829991410,3cyclictest363-21cat07:35:003
2082999140,1cyclictest0-21swapper/309:10:273
2082699142,7cyclictest0-21swapper/211:21:392
20826991413,1cyclictest0-21swapper/212:19:242
20826991411,2cyclictest31720-21cron08:40:012
20826991410,3cyclictest6940-21ssh12:39:572
20826991410,2cyclictest19953-21grep08:15:142
2082699140,13cyclictest0-21swapper/209:10:112
20824991410,3cyclictest5066-21grep11:20:161
20824991410,3cyclictest20782-21tr11:10:191
2082499140,13cyclictest26168-21cut10:35:131
2082499140,12cyclictest6010-21sed08:50:251
2082099147,6cyclictest24413-21awk08:25:000
20820991411,2cyclictest2521-21wc10:20:210
20820991410,2cyclictest29454-21cron11:55:010
2082999139,3cyclictest22012-21cat07:10:193
2082999131,2cyclictest0-21swapper/311:43:533
20829991310,2cyclictest7653-21wc07:50:013
20829991310,2cyclictest16748-21grep08:05:243
20829991310,2cyclictest1302-21sed08:40:273
20829991310,2cyclictest11529-21wc10:45:183
2082999130,6cyclictest0-21swapper/311:20:053
2082999130,12cyclictest11038-21sed09:00:253
2082699139,4cyclictest0-21swapper/211:53:062
2082699131,7cyclictest0-21swapper/209:05:132
20826991310,2cyclictest31241-21tune2fs07:30:162
20826991310,2cyclictest30299-21cron07:29:592
20826991310,2cyclictest27538-21cat11:15:002
20826991310,2cyclictest24777-21cat10:15:002
20826991310,2cyclictest23343-21cat08:20:202
20826991310,2cyclictest22864-21cat12:30:002
20826991310,2cyclictest16959-21cat09:30:262
2082699130,5cyclictest17037-21gltestperf09:50:172
2082699130,12cyclictest22804-21ssh11:30:262
2082499137,5cyclictest1232-21tail07:35:161
2082499132,1cyclictest13885-21diskmemload12:20:361
20824991311,1cyclictest22571-21aten2_r7power_e12:10:121
2082499131,11cyclictest0-21swapper/111:45:311
20824991310,2cyclictest9945-21cron10:45:001
20824991310,2cyclictest4210-21grep11:00:191
20824991310,2cyclictest23653-21grep11:50:261
20824991310,2cyclictest22265-21grep11:30:211
20824991310,1cyclictest0-21swapper/109:36:101
2082099139,3cyclictest26564-21head07:20:170
2082099139,2cyclictest23284-21taskset12:30:120
2082099136,5cyclictest13-21ksoftirqd/012:04:440
2082099133,5cyclictest32016-21apt-get08:40:130
2082099133,5cyclictest13885-21diskmemload10:30:010
2082099133,3cyclictest0-21swapper/009:50:170
2082099131,7cyclictest17261-21awk08:10:000
20820991311,1cyclictest0-21swapper/011:55:130
20820991310,2cyclictest6509-21sed11:40:220
20820991310,2cyclictest4386-21ssh11:00:210
20820991310,2cyclictest21663-21grep11:30:180
20820991310,2cyclictest19618-21latency_hist08:15:010
20820991310,2cyclictest12367-21tail10:45:250
2082099130,2cyclictest141rcu_preempt12:05:410
2082999129,2cyclictest9130-21tr09:00:013
2082999129,2cyclictest8485-21cut07:50:143
2082999129,2cyclictest5073-21grep08:50:153
2082999129,2cyclictest30830-21cut08:35:213
2082999129,2cyclictest22538-21grep08:20:153
2082999129,2cyclictest15954-21tail09:30:183
2082999129,2cyclictest12637-21grep08:00:133
2082999128,2cyclictest26593-21sed07:20:183
2082999120,1cyclictest13885-21diskmemload11:33:543
2082699129,2cyclictest7658-21grep09:25:162
2082699129,2cyclictest7636-21grep08:55:182
2082699129,2cyclictest5523-21tr08:50:192
2082699129,2cyclictest328-21cut08:40:172
2082699129,2cyclictest27103-21grep08:30:142
2082699129,2cyclictest27084-21grep07:20:222
2082699129,2cyclictest25378-21grep08:25:172
2082699129,2cyclictest25253-21sed12:30:252
2082699129,2cyclictest17777-21grep08:10:142
2082699128,3cyclictest30287-21tr11:55:142
2082699127,3cyclictest19290-21ssh10:50:152
2082699125,4cyclictest3567-21apache210:45:102
20826991211,1cyclictest0-21swapper/211:19:352
2082699120,12cyclictest0-21swapper/211:45:562
2082499129,2cyclictest608-21tr10:00:141
2082499129,2cyclictest3185-21grep08:45:211
2082499129,2cyclictest135392sleep111:05:411
2082499128,3cyclictest21992-21cron08:20:001
2082499128,3cyclictest16407-21ls12:25:221
2082499128,2cyclictest9429-21grep07:50:241
2082499128,2cyclictest24161-21grep07:15:151
2082499127,4cyclictest5417-21basename07:45:141
2082499125,7cyclictest38150irq/131-snd_hda10:52:371
2082499121,9cyclictest19329-21ssh10:10:451
20824991211,1cyclictest13885-21diskmemload11:59:291
2082499120,2cyclictest0-21swapper/111:37:331
2082499120,11cyclictest0-21swapper/112:38:471
2082099129,2cyclictest6965-21grep07:45:270
2082099129,2cyclictest3702-21grep07:40:150
2082099129,2cyclictest32594-21ssh09:20:240
2082099129,2cyclictest30457-21cat07:30:000
2082099129,2cyclictest29857-21fschecks_count11:35:160
2082099129,2cyclictest29336-21grep11:15:210
2082099129,2cyclictest27137-21grep08:30:150
2082099129,2cyclictest23162-21sh07:14:590
2082099129,2cyclictest22771-21grep12:10:130
2082099129,2cyclictest21898-21chrt08:18:500
2082099129,2cyclictest14357-21sh09:10:120
2082099129,2cyclictest13522-21grep11:45:120
2082099129,2cyclictest11073-21head09:00:260
2082099128,3cyclictest17604-21ssh10:10:150
2082099120,5cyclictest0-21swapper/011:29:130
2082999118,2cyclictest9736-21wc10:05:183
2082999118,2cyclictest3832-21ssh09:42:123
2082999118,2cyclictest27710-21grep08:30:183
2082999118,2cyclictest26271-21sed08:25:263
2082999118,2cyclictest2456-21df08:45:153
2082999118,2cyclictest23713-21grep07:15:133
2082999118,2cyclictest20021-21grep08:15:143
2082999118,2cyclictest18067-21tr10:30:143
2082999118,2cyclictest1367-21ssh10:59:123
2082999118,2cyclictest10889-21grep07:55:173
2082999110,1cyclictest0-21swapper/310:02:403
2082699119,1cyclictest0-21swapper/211:04:172
2082699118,2cyclictest7962-21grep07:50:122
2082699118,2cyclictest6595-21fschecks_count12:00:172
2082699118,2cyclictest6150-21taskset10:41:582
2082699118,2cyclictest5562-21cat07:45:152
2082699118,2cyclictest5526-21cat11:40:152
2082699118,2cyclictest3503-21ssh09:41:592
2082699118,2cyclictest3091-21tr08:45:202
2082699118,2cyclictest29083-21ssh11:35:102
2082699118,2cyclictest25908-21grep10:15:162
2082699118,2cyclictest24250-21tail09:35:162
2082699118,2cyclictest22988-21tr09:15:152
2082699118,2cyclictest1905-21grep07:35:212
2082699118,2cyclictest18399-21ssh12:07:092
2082699118,2cyclictest13206-21grep11:05:262
2082699118,2cyclictest11042-21uname07:55:182
2082699111,6cyclictest0-21swapper/207:10:132
2082699110,11cyclictest0-21swapper/210:31:202
2082699110,10cyclictest14303-21cat11:25:252
2082499118,2cyclictest3968-21ssh09:23:051
2082499118,2cyclictest32749-21tail08:40:171
2082499118,2cyclictest32182-21sed07:30:241
2082499118,2cyclictest32170-21ssh10:19:221
2082499118,2cyclictest23150-21sh12:30:061
2082499118,2cyclictest14284-21wc11:25:251
2082499118,2cyclictest14067-21cat09:10:001
2082499118,2cyclictest1369-21tr10:20:141
2082499118,2cyclictest10885-21tail07:55:171
2082499118,2cyclictest10330-21cut10:25:171
2082499117,3cyclictest17814-21ssh09:11:351
2082499114,6cyclictest38150irq/131-snd_hda07:25:201
2082499111,9cyclictest0-21swapper/112:05:131
2082099118,2cyclictest9471-21sed10:05:170
2082099118,2cyclictest9072-21ssh09:25:330
2082099118,2cyclictest8520-21ssh11:22:130
2082099118,2cyclictest7775-21head08:55:170
2082099118,2cyclictest4343-21ssh10:01:580
2082099118,2cyclictest31710-21ssh12:35:120
2082099118,2cyclictest31131-21ssh10:57:070
2082099118,2cyclictest26093-21ssh10:35:130
2082099118,2cyclictest25878-21head09:55:220
2082099118,2cyclictest25327-21cut08:25:170
2082099118,2cyclictest24140-21head07:15:160
2082099118,2cyclictest19783-21tr11:10:130
2082099118,2cyclictest19356-21ssh10:30:240
2082099118,2cyclictest15754-21grep12:25:170
2082099118,2cyclictest13374-21grep09:05:250
2082099118,2cyclictest11985-21grep11:05:160
2082099118,2cyclictest10843-21grep07:55:160
2082099118,2cyclictest10483-21ssh12:21:490
2082099112,4cyclictest0-21swapper/009:35:180
2082999108,1cyclictest5035-21taskset07:44:243
2082999108,1cyclictest29627-21cat07:25:263
2082999108,1cyclictest194132sleep308:12:323
2082999108,1cyclictest12165-21cat09:05:143
2082999107,2cyclictest768-21grep07:35:123
2082699108,1cyclictest9977-21grep09:00:152
2082699108,1cyclictest24573-21ntpq07:15:182
2082699108,1cyclictest12916-21expr08:00:152
2082699107,2cyclictest896-21ssh10:20:042
2082699100,9cyclictest8513-21ssh10:05:082
2082499108,1cyclictest289342sleep108:32:281
2082499108,1cyclictest25812-21grep08:25:211
2082499108,1cyclictest169012sleep108:05:441
2082499107,2cyclictest27485-21tail07:20:271
2082499107,2cyclictest11128-21chrt09:00:301
2082099108,1cyclictest9629-21taskset07:52:270
2082099107,2cyclictest31718-21cron08:40:010
2082099107,2cyclictest31124-21grep07:30:140
2082099107,2cyclictest14571-21taskset08:04:400
2082099104,5cyclictest1235-21ntpd08:45:200
2082099100,9cyclictest30993-21wc12:15:140
208249997,1cyclictest18715-21cut08:10:201
208249996,2cyclictest21825-21tail07:10:171
208249995,3cyclictest12735-21tr08:00:151
208209996,2cyclictest528-21aten2_r7power_c07:35:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional