You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-31 - 14:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Tue Mar 31, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1044021050,0sleep20-21swapper/211:16:592
3048229481,9sleep00-21swapper/007:07:530
2852128471,9sleep20-21swapper/207:05:112
2856028365,13sleep10-21swapper/107:05:131
3044928168,9sleep30-21swapper/307:07:293
3073599280,28cyclictest0-21swapper/209:16:592
3072999270,26cyclictest0-21swapper/011:34:400
3072999263,1cyclictest464-21apt-get09:35:020
3072999252,3cyclictest0-21swapper/012:29:540
3072999250,1cyclictest0-21swapper/010:35:140
3073999242,1cyclictest0-21swapper/311:50:363
3072999242,2cyclictest23533-21diskmemload11:57:410
3073399230,3cyclictest0-21swapper/111:30:171
3072999230,4cyclictest0-21swapper/011:11:480
3073599221,0cyclictest0-21swapper/210:40:142
3072999222,2cyclictest23533-21diskmemload09:53:070
3072999220,1cyclictest0-21swapper/011:17:440
3072999220,1cyclictest0-21swapper/010:18:140
263262220,2sleep226323-21apache_accesses12:25:122
30729992114,7cyclictest0-21swapper/012:31:450
3072999211,1cyclictest0-21swapper/010:10:120
3072999210,21cyclictest0-21swapper/010:04:300
3072999210,14cyclictest0-21swapper/010:30:130
3073999200,1cyclictest0-21swapper/309:20:143
3073399209,7cyclictest452-21systemd-logind08:00:221
3073399209,7cyclictest452-21systemd-logind08:00:221
3073399200,1cyclictest0-21swapper/112:00:461
3073399200,1cyclictest0-21swapper/111:11:371
3072999200,5cyclictest0-21swapper/012:10:130
3072999200,2cyclictest0-21swapper/012:04:400
3072999200,20cyclictest0-21swapper/009:24:150
3072999200,0cyclictest0-21swapper/012:22:400
3073599190,1cyclictest0-21swapper/210:39:452
3073399190,2cyclictest0-21swapper/111:53:221
3073399190,19cyclictest23533-21diskmemload09:54:221
3072999196,6cyclictest879-21perf07:15:000
3072999191,3cyclictest23533-21diskmemload10:27:410
3072999191,3cyclictest23533-21diskmemload09:17:290
3072999191,1cyclictest225-21systemd-journal09:14:590
3072999191,1cyclictest0-21swapper/010:49:140
3072999190,2cyclictest0-21swapper/012:15:050
3073999180,1cyclictest0-21swapper/310:09:583
3073599182,14cyclictest0-21swapper/210:06:092
30735991816,1cyclictest141rcu_preempt09:35:032
3073599180,0cyclictest0-21swapper/212:10:212
30733991818,0cyclictest0-21swapper/110:52:201
3072999188,5cyclictest11607-21ssh10:20:110
3072999187,7cyclictest5361-21taskset07:21:290
3072999183,2cyclictest23533-21diskmemload10:53:400
3072999181,3cyclictest10426-21/usr/sbin/munin12:35:190
3072999180,2cyclictest0-21swapper/011:22:430
30735991711,2cyclictest9913-21ntpq11:55:202
3073399170,16cyclictest0-21swapper/112:39:261
3072999172,1cyclictest23533-21diskmemload11:27:260
3072999170,3cyclictest0-21swapper/009:27:570
3072999170,1cyclictest25420-21cut12:05:130
3073999160,6cyclictest0-21swapper/312:29:353
3073399161,14cyclictest0-21swapper/107:45:131
3073399160,16cyclictest0-21swapper/109:32:491
3072999160,15cyclictest25646-21awk11:45:190
30739991511,3cyclictest11182-21grep12:35:153
3073999150,15cyclictest0-21swapper/309:31:203
30735991514,1cyclictest291irq_work/210:29:482
30735991513,2cyclictest0-21swapper/209:30:452
30735991512,2cyclictest0-21swapper/211:22:032
3073599150,1cyclictest0-21swapper/211:13:122
3073599150,0cyclictest0-21swapper/210:49:572
3073399152,12cyclictest0-21swapper/110:35:121
30733991514,1cyclictest0-21swapper/111:16:291
30733991510,3cyclictest20172-21cron07:55:001
30733991510,3cyclictest16369-21cron08:55:001
3073399150,15cyclictest0-21swapper/112:30:241
3072999159,5cyclictest30209-21cut08:15:160
3072999159,5cyclictest18296-21tr09:45:140
3072999159,5cyclictest1039-21tail11:50:170
3072999158,6cyclictest8032-21wc08:35:180
3072999155,1cyclictest0-21swapper/010:44:310
3072999152,2cyclictest225-21systemd-journal09:45:000
3072999152,2cyclictest16190-21df_abs07:45:150
30729991510,4cyclictest13-21ksoftirqd/009:30:220
3072999150,4cyclictest7021-21cut07:25:190
3072999150,3cyclictest0-21swapper/011:39:360
30739991412,1cyclictest23533-21diskmemload09:59:443
30739991411,2cyclictest16473-21taskset11:00:513
30739991410,3cyclictest14185-21tr07:40:213
30739991410,2cyclictest29240-21grep08:10:263
3073999140,1cyclictest0-21swapper/309:38:503
3073999140,13cyclictest22553-21idleruntime-cro08:00:013
30735991411,2cyclictest32-21ksoftirqd/207:50:112
30735991411,2cyclictest26828-21grep08:05:252
3073599140,12cyclictest0-21swapper/211:32:432
3073399142,11cyclictest10662-21sh10:57:551
30733991412,1cyclictest14904-21apache212:24:451
3073399141,12cyclictest2673-21ssh10:14:581
3073399140,1cyclictest0-21swapper/111:41:121
3073399140,14cyclictest0-21swapper/111:59:511
3072999149,4cyclictest658-21grep08:20:170
3072999149,4cyclictest22984-21sort11:05:160
3072999148,5cyclictest21160-21grep07:55:160
3072999148,5cyclictest17973-21ssh11:40:250
3072999147,6cyclictest7279-21ssh09:57:480
3072999147,6cyclictest20099-21cat10:05:190
3072999147,6cyclictest16942-21sh11:01:130
3072999147,6cyclictest13323-21wc08:45:260
3072999144,9cyclictest29711-21munin-run08:15:000
30729991410,3cyclictest6365-21cut10:55:170
30729991410,3cyclictest3431-21grep08:25:210
3073999139,3cyclictest3269-21perf07:20:003
3073999139,3cyclictest32669-21cut09:15:143
3073999139,3cyclictest12554-21ssh09:41:113
3073999138,3cyclictest9733-21grep12:15:143
30739991311,2cyclictest39-21ksoftirqd/311:47:093
30739991310,2cyclictest6694-21chrt10:35:373
30739991310,2cyclictest28695-21ssh09:50:423
30739991310,2cyclictest2784-21ssh10:15:003
30739991310,2cyclictest22977-21grep08:00:143
30739991310,2cyclictest22977-21grep08:00:133
30739991310,2cyclictest19759-21sed07:50:263
30739991310,2cyclictest18014-21dpkg12:20:133
30739991310,2cyclictest12756-21cron07:40:003
3073599139,3cyclictest3598-21cut10:15:162
3073599137,5cyclictest14904-21apache212:07:002
30735991310,2cyclictest5007-21awk10:55:002
30735991310,2cyclictest3101-21head09:55:172
30735991310,2cyclictest24185-21grep11:05:262
30735991310,2cyclictest23959-21uniq11:25:162
30735991310,2cyclictest2106-21sed11:50:252
30735991310,2cyclictest15934-21grep07:45:152
30735991310,2cyclictest14132-21grep08:50:002
30735991310,2cyclictest10694-21tr08:40:212
3073599130,1cyclictest0-21swapper/212:35:212
3073599130,13cyclictest0-21swapper/211:39:532
3073399139,3cyclictest5812-21cut08:30:211
3073399133,6cyclictest225-21systemd-journal12:05:171
30733991313,0cyclictest0-21swapper/109:48:131
30733991310,2cyclictest7777-21grep08:35:161
30733991310,2cyclictest614-21cut08:20:161
30733991310,2cyclictest30417-21cat10:30:251
30733991310,2cyclictest28244-21sed12:25:261
30733991310,2cyclictest17546-21grep09:25:181
3073399130,1cyclictest0-21swapper/109:22:261
3073399130,1cyclictest0-21swapper/109:18:241
3073399130,13cyclictest0-21swapper/110:29:591
3073399130,12cyclictest1690-21grep07:15:161
3072999138,4cyclictest14349-21cut08:50:120
3072999138,4cyclictest12519-21chrt07:36:580
3072999137,5cyclictest19658-21fschecks_count09:00:130
3072999137,5cyclictest11550-21taskset08:44:520
3072999136,6cyclictest16928-21tr08:55:140
3072999135,2cyclictest14097-21cat07:40:190
3072999132,3cyclictest22335-21irqstats09:05:200
3072999132,2cyclictest0-21swapper/007:15:140
30729991310,2cyclictest9798-21wc07:30:260
3072999130,4cyclictest225-21systemd-journal08:00:230
3072999130,4cyclictest225-21systemd-journal08:00:220
256632130,0chrt0-21swapper/311:06:303
3073999129,2cyclictest8161-21tail08:35:203
3073999129,2cyclictest6166-21sort10:55:173
3073999129,2cyclictest3994-21sort10:15:173
3073999129,2cyclictest32511-21uname11:30:173
3073999129,2cyclictest31692-21sed08:15:283
3073999129,2cyclictest26810-21sh12:05:223
3073999129,2cyclictest17528-21grep09:25:173
3073999129,2cyclictest16614-21sed11:40:143
3073999129,2cyclictest16102-21grep11:20:203
3073999128,3cyclictest4856-21find07:20:203
3073999128,3cyclictest12616-21grep08:45:173
3073999128,3cyclictest12535-21sed10:00:273
3073999127,4cyclictest2365-21kernelversion12:10:213
3073999126,3cyclictest0-21swapper/309:49:333
3073999124,4cyclictest23533-21diskmemload10:50:193
3073999122,7cyclictest8286-21apache_processe07:30:133
3073999120,5cyclictest0-21swapper/311:29:023
3073599129,2cyclictest29278-21grep10:30:162
3073599129,2cyclictest23852-21cron11:44:592
3073599129,2cyclictest19678-21apt-config10:25:012
3073599129,2cyclictest13183-21basename07:40:142
3073599129,2cyclictest10847-21head09:40:202
3073599128,3cyclictest7801-21grep08:35:172
30735991210,1cyclictest11616-21awk12:15:272
30735991210,1cyclictest0-21swapper/208:25:162
3073599120,9cyclictest0-21swapper/212:02:152
3073599120,6cyclictest10202-21latency_hist12:35:002
3073599120,11cyclictest26238-21grep11:45:262
3073399129,2cyclictest4591-21cat07:20:181
3073399129,2cyclictest4084-21grep10:15:181
3073399129,2cyclictest32103-21basename08:20:011
3073399129,2cyclictest18755-21ls08:59:591
3073399129,2cyclictest17467-21cron12:20:001
3073399129,2cyclictest16385-21wc11:20:211
3073399127,4cyclictest20367-21cat09:00:211
3073399127,4cyclictest17695-21ssh11:01:491
3073399120,2cyclictest0-21swapper/110:23:301
3073399120,1cyclictest0-21swapper/109:35:381
3073399120,1cyclictest0-21swapper/108:29:391
3073399120,11cyclictest19857-21grep10:05:171
3073399120,11cyclictest0-21swapper/111:38:351
3072999129,2cyclictest26318-21grep08:05:200
3072999126,2cyclictest19273-21cut07:50:210
3072999120,3cyclictest225-21systemd-journal08:33:510
3073999118,2cyclictest7298-21head11:15:163
3073999118,2cyclictest6405-21ssh12:32:063
3073999118,2cyclictest32592-21ssh11:10:293
3073999118,2cyclictest31829-21uniq07:10:153
3073999118,2cyclictest25664-21cat08:05:143
3073999118,2cyclictest21102-21ssh10:45:093
3073999118,2cyclictest20485-21tr10:25:133
3073999118,2cyclictest17257-21egrep12:00:143
3073999118,2cyclictest1597-21ssh10:33:083
3073999118,2cyclictest13744-21ssh11:57:543
3073999118,2cyclictest13152-21mailstats10:20:223
3073999118,2cyclictest12682-21ssh10:40:083
3073999118,2cyclictest11195-21ssh11:36:513
3073999117,3cyclictest15983-21tr07:45:143
3073999113,2cyclictest0-21swapper/309:05:003
3073999111,1cyclictest0-21swapper/309:10:563
3073599119,1cyclictest22919-21chrt12:22:422
3073599118,2cyclictest9108-21grep09:20:162
3073599118,2cyclictest7912-21taskset07:28:502
3073599118,2cyclictest7487-21ssh10:55:272
3073599118,2cyclictest669-21grep08:20:172
3073599118,2cyclictest5614-21taskset07:24:462
3073599118,2cyclictest31585-21grep08:15:272
3073599118,2cyclictest3090-21taskset07:18:082
3073599118,2cyclictest26622-21cut09:50:142
3073599118,2cyclictest24070-21grep08:00:222
3073599118,2cyclictest24070-21grep08:00:212
3073599118,2cyclictest23189-21seq09:05:532
3073599118,2cyclictest19306-21ssh09:25:542
3073599118,2cyclictest17803-21ssh09:45:062
30735991110,1cyclictest27370-21ssh10:10:132
3073399119,1cyclictest9777-21wget09:40:151
3073399118,2cyclictest3732-21chrt12:10:471
3073399118,2cyclictest3340-21ssh09:55:191
3073399118,2cyclictest24067-21head11:25:181
3073399118,2cyclictest21704-21wc07:55:221
3073399118,2cyclictest15074-21sh10:02:201
3073399118,2cyclictest15023-21ssh10:40:331
3073399118,2cyclictest11121-21grep08:40:261
3073399118,2cyclictest10884-21grep07:35:151
3073399117,3cyclictest13814-21taskset08:48:281
3073399117,2cyclictest15385-21wc07:45:001
3073399113,6cyclictest22243-21/usr/sbin/munin11:05:201
3073399110,10cyclictest28370-21ssh11:47:011
3073399110,10cyclictest27431-21ssh09:11:311
3073399110,10cyclictest22012-21grep10:45:161
3073999108,1cyclictest70042sleep307:25:193
3073999108,1cyclictest4454-21basename08:30:013
3073999108,1cyclictest21861-21expr09:05:153
3073999108,1cyclictest1759-21taskset08:21:183
3073999108,1cyclictest10820-21cut08:40:223
3073999107,2cyclictest4885-21egrep08:30:153
3073999107,2cyclictest17349-21grep08:55:163
3073999107,2cyclictest14501-21cut08:50:123
3073599108,1cyclictest20918-21cat07:55:152
3073599108,1cyclictest17026-21expr08:55:142
3073599108,1cyclictest15210-21cut08:50:182
3073599108,1cyclictest13639-21sh11:00:032
3073599107,2cyclictest31147-21taskset07:10:092
3073599107,2cyclictest27587-21seq08:10:122
3073599107,2cyclictest24492-21expr09:10:132
3073599107,2cyclictest19376-21expr09:00:132
3073599107,2cyclictest10520-21rm10:00:102
3073599107,2cyclictest10051-21chrt07:30:442
3073599100,9cyclictest12156-21cat07:35:262
3073399108,1cyclictest270572sleep108:06:451
3073399108,1cyclictest21660-21expr09:05:151
3073399107,2cyclictest9890-21unixbench-2d07:30:271
3073399107,2cyclictest31424-21grep07:10:131
3073399107,2cyclictest27830-21grep08:10:141
307359996,2cyclictest4977-21cut08:30:152
307339996,2cyclictest6288-21expr07:25:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional