You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-29 - 23:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Wed Apr 29, 2026 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15551992522,2cyclictest11841-21sh10:45:583
1555199250,25cyclictest9242-21diskmemload11:12:373
1555199250,24cyclictest4208-21sh12:13:003
1555199250,24cyclictest27093-21timerwakeupswit09:26:013
1555099250,24cyclictest10003-21awk08:05:012
1555099250,22cyclictest0-21swapper/209:15:572
1554699250,24cyclictest12083-21sh11:17:511
1554699250,24cyclictest0-21swapper/110:44:021
1554699250,24cyclictest0-21swapper/109:30:531
1554699250,24cyclictest0-21swapper/109:30:531
1554699250,1cyclictest11511-21ssh12:20:531
1554699250,1cyclictest0-21swapper/111:37:461
1554499250,24cyclictest0-21swapper/011:35:040
1554499250,24cyclictest0-21swapper/011:33:540
1554499250,24cyclictest0-21swapper/010:34:040
1554499250,1cyclictest9242-21diskmemload09:30:530
1554499250,1cyclictest9242-21diskmemload09:30:530
15551992423,1cyclictest9299-21ssh10:44:023
15551992423,1cyclictest21314-21cron12:30:003
1555199240,24cyclictest0-21swapper/311:56:173
1555199240,23cyclictest0-21swapper/312:20:533
1555199240,23cyclictest0-21swapper/312:00:573
1555199240,23cyclictest0-21swapper/311:37:463
1555099240,24cyclictest15540-21cyclictest10:04:532
1555099240,22cyclictest25012-21ssh09:25:072
15546992423,1cyclictest0-21swapper/110:45:041
15546992423,0cyclictest9242-21diskmemload11:42:091
15546992423,0cyclictest0-21swapper/111:07:041
15546992422,1cyclictest9242-21diskmemload12:30:521
1554699240,24cyclictest0-21swapper/109:21:031
1554699240,23cyclictest28749-21kworker/u8:210:08:431
1554699240,23cyclictest0-21swapper/109:44:471
1554699240,1cyclictest0-21swapper/112:00:571
1554699240,1cyclictest0-21swapper/109:26:011
1554699240,0cyclictest0-21swapper/111:53:531
1554699240,0cyclictest0-21swapper/111:53:531
15544992423,1cyclictest0-21swapper/010:59:020
1554499240,23cyclictest0-21swapper/012:11:420
1554499240,23cyclictest0-21swapper/011:49:110
1554499240,23cyclictest0-21swapper/011:28:140
1554499240,22cyclictest167802chrt12:25:380
1554499240,1cyclictest0-21swapper/012:30:520
15551992323,0cyclictest0-21swapper/311:53:533
15551992323,0cyclictest0-21swapper/311:53:533
15551992323,0cyclictest0-21swapper/309:48:293
15551992323,0cyclictest0-21swapper/309:30:533
15551992323,0cyclictest0-21swapper/309:30:533
15551992323,0cyclictest0-21swapper/309:21:033
15551992322,1cyclictest9242-21diskmemload10:38:083
15551992322,1cyclictest27652-21ssh11:32:203
15551992322,1cyclictest0-21swapper/311:42:093
15551992320,2cyclictest29690-21sh10:00:553
15551992320,2cyclictest21283-21ssh10:24:483
1555199230,23cyclictest0-21swapper/312:06:083
1555199230,23cyclictest0-21swapper/311:17:513
1555199230,22cyclictest9242-21diskmemload11:47:003
1555199230,22cyclictest9242-21diskmemload10:18:473
1555199230,22cyclictest9242-21diskmemload10:07:323
1555199230,22cyclictest29031-21ssh11:02:393
1555199230,22cyclictest27240-21sendmail08:40:003
1555199230,21cyclictest30006-21awk07:40:003
1555199230,0cyclictest0-21swapper/312:30:053
15550992323,0cyclictest9242-21diskmemload09:54:412
15550992323,0cyclictest0-21swapper/212:35:032
15550992323,0cyclictest0-21swapper/212:32:462
15550992323,0cyclictest0-21swapper/211:06:042
15550992323,0cyclictest0-21swapper/209:33:162
15550992323,0cyclictest0-21swapper/209:33:162
15550992322,1cyclictest9242-21diskmemload09:48:462
15550992322,1cyclictest3952-21apt-get11:10:522
1555099230,3cyclictest22294-21ssh09:21:102
1555099230,2cyclictest10860-21rm11:47:552
1555099230,23cyclictest0-21swapper/211:44:312
1555099230,23cyclictest0-21swapper/210:38:382
1555099230,22cyclictest9547-21ssh10:12:282
1555099230,1cyclictest25355-21ssh12:01:242
1555099230,0cyclictest9242-21diskmemload10:45:452
1555099230,0cyclictest0-21swapper/211:29:502
1555099230,0cyclictest0-21swapper/211:01:072
1555099230,0cyclictest0-21swapper/210:20:132
15546992323,0cyclictest9242-21diskmemload10:31:561
15546992323,0cyclictest0-21swapper/109:50:041
15546992321,1cyclictest0-21swapper/111:46:031
15546992321,1cyclictest0-21swapper/111:25:531
15546992321,1cyclictest0-21swapper/111:10:501
15546992321,1cyclictest0-21swapper/110:51:001
15546992321,1cyclictest0-21swapper/110:01:041
15546992321,1cyclictest0-21swapper/109:57:361
1554699230,23cyclictest0-21swapper/111:02:281
1554699230,23cyclictest0-21swapper/110:27:341
1554699230,22cyclictest9242-21diskmemload12:16:521
1554699230,22cyclictest27278-21wc08:40:001
1554699230,22cyclictest22894-21sh10:56:061
1554699230,22cyclictest0-21swapper/112:11:121
1554699230,22cyclictest0-21swapper/110:38:081
1554699230,0cyclictest9242-21diskmemload09:46:441
15544992323,0cyclictest0-21swapper/011:02:230
15544992323,0cyclictest0-21swapper/009:35:520
15544992322,0cyclictest9242-21diskmemload11:42:280
15544992321,1cyclictest0-21swapper/009:15:270
15544992320,1cyclictest0-21swapper/009:52:260
1554499230,23cyclictest0-21swapper/010:05:300
1554499230,23cyclictest0-21swapper/010:03:300
1554499230,23cyclictest0-21swapper/009:45:050
1554499230,22cyclictest9242-21diskmemload12:15:080
1554499230,22cyclictest28100-21df_inode12:35:530
1554499230,22cyclictest26543-21ntp_states09:25:570
1554499230,22cyclictest12747-21sh10:46:140
1554499230,22cyclictest0-21swapper/012:00:590
1554499230,22cyclictest0-21swapper/011:55:060
1554499230,22cyclictest0-21swapper/010:22:530
1554499230,22cyclictest0-21swapper/009:43:260
1554499230,1cyclictest0-21swapper/011:13:110
1554499230,0cyclictest9242-21diskmemload09:20:480
1554499230,0cyclictest0-21swapper/009:59:000
15551992222,0cyclictest0-21swapper/312:35:263
15551992222,0cyclictest0-21swapper/311:05:503
15551992220,1cyclictest9242-21diskmemload09:35:403
1555199220,22cyclictest0-21swapper/311:21:523
1555199220,21cyclictest9242-21diskmemload12:15:483
1555199220,21cyclictest30347-21apt-get07:40:503
1555199220,21cyclictest2414-21perf07:50:013
1555199220,21cyclictest21811-21sh11:26:183
1555199220,21cyclictest14859-21ssh09:15:193
1555199220,21cyclictest0-21swapper/310:56:063
1555199220,21cyclictest0-21swapper/310:34:253
1555199220,1cyclictest22374-21ssh10:25:283
1555199220,0cyclictest18300-21ssh10:51:583
1555199220,0cyclictest0-21swapper/309:55:153
15550992222,0cyclictest0-21swapper/211:16:512
15550992222,0cyclictest0-21swapper/210:25:222
15550992222,0cyclictest0-21swapper/209:10:152
15550992221,1cyclictest5576-21ssh09:36:502
15550992220,1cyclictest14852-21wc08:15:002
15550992219,1cyclictest8132-21sh09:40:292
1555099220,22cyclictest0-21swapper/210:06:332
1555099220,21cyclictest29684-21latency_hist08:45:012
1555099220,21cyclictest29512-21unixbench_singl10:30:592
1555099220,21cyclictest26916-21ssh11:31:142
1555099220,21cyclictest22802-21users10:56:002
1555099220,21cyclictest16542-21ssh12:25:082
1555099220,21cyclictest15059-21ssh12:23:542
1555099220,21cyclictest12950-21sh11:50:442
1555099220,21cyclictest12950-21sh11:50:442
1555099220,21cyclictest0-21swapper/212:05:112
1555099220,21cyclictest0-21swapper/211:55:512
1555099220,21cyclictest0-21swapper/210:50:512
1555099220,21cyclictest0-21swapper/207:30:482
1555099220,1cyclictest12421-21ssh10:15:522
1555099220,1cyclictest0-21swapper/212:10:512
1555099220,1cyclictest0-21swapper/209:59:282
15546992222,0cyclictest9242-21diskmemload10:16:111
15546992221,1cyclictest0-21swapper/111:33:591
15546992220,1cyclictest32074-21turbostat.cron08:50:011
15546992220,1cyclictest0-21swapper/112:26:141
15546992220,1cyclictest0-21swapper/111:21:101
1554699221,21cyclictest609-21rs:main1
1554699220,22cyclictest19246-21ssh11:55:561
1554699220,21cyclictest26436-21apt-get12:35:021
1554699220,21cyclictest16814-21ssh10:20:211
1554699220,21cyclictest0-21swapper/112:05:551
1554699220,21cyclictest0-21swapper/109:10:081
1554699220,21cyclictest0-21swapper/107:35:511
1554699220,1cyclictest0-21swapper/110:10:151
1554699220,1cyclictest0-21swapper/109:38:391
1554699220,1cyclictest0-21swapper/109:15:191
1554699220,1cyclictest0-21swapper/107:50:011
15544992222,0cyclictest0-21swapper/012:05:320
15544992221,0cyclictest0-21swapper/011:20:050
15544992221,0cyclictest0-21swapper/008:30:500
15544992220,2cyclictest0-21swapper/011:50:330
15544992220,2cyclictest0-21swapper/011:50:330
15544992220,1cyclictest12131-21tr10:15:510
15544992220,1cyclictest0-21swapper/011:15:160
15544992220,1cyclictest0-21swapper/010:50:170
15544992220,1cyclictest0-21swapper/010:10:510
1554499220,22cyclictest0-21swapper/011:05:540
1554499220,21cyclictest9242-21diskmemload10:25:040
1554499220,21cyclictest606-21dbus-daemon08:00:010
1554499220,21cyclictest27522-21munin-run07:35:000
1554499220,21cyclictest0-21swapper/012:22:410
1554499220,21cyclictest0-21swapper/010:41:090
1554499220,21cyclictest0-21swapper/008:45:510
1554499220,21cyclictest0-21swapper/008:40:510
1554499220,21cyclictest0-21swapper/007:45:500
1554499220,21cyclictest0-21swapper/007:15:500
1554499220,1cyclictest2397-21ssh10:36:110
1554499220,0cyclictest0-21swapper/009:12:120
1554499220,0cyclictest0-21swapper/009:05:510
15551992121,0cyclictest0-21swapper/309:50:043
15551992121,0cyclictest0-21swapper/309:40:303
15551992119,2cyclictest27521-21latency_hist07:35:003
1555199210,21cyclictest6402-21sh10:10:153
1555199210,20cyclictest17916-21perf07:15:013
1555199210,20cyclictest0-21swapper/309:10:403
1555199210,20cyclictest0-21swapper/307:55:503
15550992121,0cyclictest5173-21ssh10:40:092
15550992120,1cyclictest32391-21apt-get08:50:512
15550992119,1cyclictest23031-21apt-get07:25:492
1555099210,21cyclictest0-21swapper/211:35:342
1555099210,21cyclictest0-21swapper/207:10:512
1555099210,20cyclictest32086-21munin-run08:50:012
1555099210,1cyclictest14728-21ssh11:20:522
1555099210,0cyclictest6225-21ssh12:15:442
15546992121,0cyclictest0-21swapper/108:15:491
1554699210,20cyclictest17268-21apt-get08:20:021
1554699210,20cyclictest0-21swapper/107:15:491
1554699210,1cyclictest5126-21apt-get07:55:501
1554699210,1cyclictest0-21swapper/107:20:521
1554699210,0cyclictest0-21swapper/107:30:481
15544992120,1cyclictest0-21swapper/008:35:500
15544992120,1cyclictest0-21swapper/008:05:510
1554499210,20cyclictest0-21swapper/007:40:500
15551992020,0cyclictest0-21swapper/308:20:023
15551992020,0cyclictest0-21swapper/308:05:503
15551992019,0cyclictest0-21swapper/308:15:493
15551992018,1cyclictest32069-21apt08:50:013
1555199200,0cyclictest7186-21apt-get09:05:513
1555199200,0cyclictest0-21swapper/307:15:503
15550992020,0cyclictest0-21swapper/208:30:502
15550992020,0cyclictest0-21swapper/207:40:502
15550992019,0cyclictest0-21swapper/207:55:502
1555099200,0cyclictest2732-21apt-get07:50:522
15546992020,0cyclictest0-21swapper/109:05:511
15546992020,0cyclictest0-21swapper/108:00:521
15546992019,0cyclictest0-21swapper/109:00:511
15544992020,0cyclictest0-21swapper/007:20:530
1554499200,0cyclictest0-21swapper/009:00:510
15551991918,0cyclictest7940-21apt-get08:00:523
15550991919,0cyclictest0-21swapper/207:20:532
15546991918,1cyclictest0-21swapper/108:40:511
15546991918,1cyclictest0-21swapper/107:40:501
15546991918,0cyclictest22790-21apt-get08:30:501
15544991919,0cyclictest0-21swapper/007:35:510
15551991817,1cyclictest27602-21apt-get08:40:513
15551991817,1cyclictest0-21swapper/309:00:513
15551991816,1cyclictest0-21swapper/308:30:503
15546991817,1cyclictest0-21swapper/108:05:511
15546991816,1cyclictest0-21swapper/108:10:511
15544991818,0cyclictest0-21swapper/008:00:020
15544991817,1cyclictest20068-21dump-pmu-power08:25:000
15544991816,1cyclictest22454-21turbostat.cron08:30:000
1555199170,16cyclictest0-21swapper/308:50:523
15550991717,0cyclictest20391-21apt-get08:25:502
15550991717,0cyclictest0-21swapper/208:20:022
15550991716,1cyclictest25184-21apt-get08:35:502
15544991716,0cyclictest0-21swapper/007:10:510
15551991614,1cyclictest12781-21apt-get08:10:513
1555199160,0cyclictest0-21swapper/307:25:493
15550991615,1cyclictest32745-21apt-get07:45:502
15544991614,1cyclictest17235-21cron08:20:000
15544991614,1cyclictest0-21swapper/008:56:000
1554499160,15cyclictest25094-21apt07:30:000
1554499160,15cyclictest13829-21ntp_states08:10:560
15551991515,0cyclictest0-21swapper/307:20:533
1555199150,4cyclictest0-21swapper/307:50:523
1555199150,1cyclictest783-21snmpd08:28:133
15550991515,0cyclictest10333-21apt-get08:05:512
15550991513,1cyclictest30013-21cron07:40:002
15544991514,0cyclictest0-21swapper/008:50:510
1555199140,0cyclictest0-21swapper/308:55:543
1555099140,14cyclictest0-21swapper/209:05:232
1555099140,14cyclictest0-21swapper/208:55:022
1555099140,14cyclictest0-21swapper/207:15:142
1555099140,0cyclictest0-21swapper/209:00:172
1555099140,0cyclictest0-21swapper/208:17:072
15546991414,0cyclictest0-21swapper/108:55:151
15546991414,0cyclictest0-21swapper/108:25:101
1554699140,14cyclictest0-21swapper/107:50:231
1554699140,0cyclictest0-21swapper/108:50:201
1554499140,0cyclictest0-21swapper/007:50:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional