You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-19 - 02:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Kontron
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot3s.osadl.org (updated Wed Mar 18, 2026 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
159242840,2sleep015918-21cron07:10:010
1560026657,6sleep30-21swapper/307:06:003
1300916556,6phc2sys0-21swapper/107:06:361
1589025248,2sleep20-21swapper/207:09:432
4552420,0sleep00-21swapper/011:40:200
119591200,0ptp4l0-21swapper/010:20:130
119591180,0ptp4l0-21swapper/107:25:181
119591170,0ptp4l0-21swapper/012:20:150
119591170,0ptp4l0-21swapper/008:30:120
119591160,1ptp4l5977-21apt-get07:50:121
119591160,1ptp4l5384-21apt-get08:50:121
119591160,1ptp4l17294-21snmp_o5powerctl10:10:220
119591160,1ptp4l1381-21snmp_o5powerctl09:40:221
119591160,1ptp4l12953-21snmp_o5powerctl08:00:211
119591160,1ptp4l12347-21apt-get11:05:020
119591160,0ptp4l0-21swapper/111:10:231
119591160,0ptp4l0-21swapper/110:15:141
119591160,0ptp4l0-21swapper/012:00:130
119591150,1ptp4l9916-21snmp_o5powerctl08:55:220
119591150,1ptp4l5639-21ntp_states11:50:181
119591150,1ptp4l4178-21apt-get10:50:021
119591150,1ptp4l4034-21snmp_o5powerctl09:45:220
119591150,1ptp4l3284-21snmp_o5powerctl11:45:220
119591150,1ptp4l29787-21smartctl12:35:201
119591150,1ptp4l27149-21snmp_o5powerctl12:30:221
119591150,1ptp4l24427-21latency_hist08:25:001
119591150,1ptp4l24105-21ls07:20:250
119591150,1ptp4l24055-21diskstats10:25:140
119591150,1ptp4l23916-21users08:20:220
119591150,1ptp4l21856-21snmp_o5powerctl12:20:221
119591150,1ptp4l21278-21snmp_o5powerctl07:15:231
119591150,1ptp4l20901-21snmp_o5powerctl08:15:221
119591150,1ptp4l15601-21snmp_o5powerctl08:05:191
119591150,1ptp4l15330-21df_inode12:10:141
119591150,1ptp4l13965-21ls12:05:221
119591150,0ptp4l0-21swapper/111:40:111
119591150,0ptp4l0-21swapper/108:30:111
119591150,0ptp4l0-21swapper/012:35:120
119591150,0ptp4l0-21swapper/011:03:280
119591150,0ptp4l0-21swapper/010:55:360
119591150,0ptp4l0-21swapper/009:25:250
119591150,0ptp4l0-21swapper/009:20:190
119591150,0ptp4l0-21swapper/008:28:320
119591150,0ptp4l0-21swapper/007:35:100
119591140,1ptp4l9089-21df_inode07:55:110
119591140,1ptp4l7666-21aten_r9power_cu09:55:121
119591140,1ptp4l7652-21snmp_o5powerctl07:50:220
119591140,1ptp4l6390-21ntp_states09:50:170
119591140,1ptp4l5380-21if_enp0s31f610:50:170
119591140,1ptp4l31282-21latency_hist10:40:020
119591140,1ptp4l30085-21head09:35:140
119591140,1ptp4l28240-21expr07:30:141
119591140,1ptp4l27065-21rt-features12:30:210
119591140,1ptp4l26809-21snmp_o5powerctl07:25:220
119591140,1ptp4l26733-21latency_hist09:30:002
119591140,1ptp4l25385-21memory09:25:181
119591140,1ptp4l24303-21/usr/sbin/munin07:20:261
119591140,1ptp4l23612-21if_enp3s012:25:151
119591140,1ptp4l19193-21snmp_o5powerctl12:15:240
119591140,1ptp4l18075-21/usr/sbin/munin11:15:143
119591140,1ptp4l17563-21cut08:10:160
119591140,1ptp4l16523-21cstates09:10:121
119591140,1ptp4l1457-21ntpq08:40:160
119591140,1ptp4l12578-21snmp_o5powerctl09:00:210
119591140,1ptp4l12015-21irqstats09:00:161
119591140,0ptp4l24262-21/usr/sbin/munin10:25:151
119591140,0ptp4l14933-21ntp_states09:05:180
119591140,0ptp4l0-21swapper/312:30:223
119591140,0ptp4l0-21swapper/211:40:172
119591140,0ptp4l0-21swapper/210:38:462
119591140,0ptp4l0-21swapper/209:24:592
119591140,0ptp4l0-21swapper/112:00:161
119591140,0ptp4l0-21swapper/111:26:291
119591140,0ptp4l0-21swapper/111:17:211
119591140,0ptp4l0-21swapper/111:05:281
119591140,0ptp4l0-21swapper/110:46:061
119591140,0ptp4l0-21swapper/110:40:121
119591140,0ptp4l0-21swapper/110:24:501
119591140,0ptp4l0-21swapper/110:00:201
119591140,0ptp4l0-21swapper/109:50:211
119591140,0ptp4l0-21swapper/108:55:011
119591140,0ptp4l0-21swapper/108:45:171
119591140,0ptp4l0-21swapper/108:40:201
119591140,0ptp4l0-21swapper/108:35:101
119591140,0ptp4l0-21swapper/108:25:181
119591140,0ptp4l0-21swapper/108:10:181
119591140,0ptp4l0-21swapper/107:40:201
119591140,0ptp4l0-21swapper/107:35:201
119591140,0ptp4l0-21swapper/107:10:001
119591140,0ptp4l0-21swapper/012:25:150
119591140,0ptp4l0-21swapper/011:55:210
119591140,0ptp4l0-21swapper/011:35:220
119591140,0ptp4l0-21swapper/011:15:210
119591140,0ptp4l0-21swapper/010:30:210
119591140,0ptp4l0-21swapper/010:15:200
119591140,0ptp4l0-21swapper/010:05:100
119591140,0ptp4l0-21swapper/009:30:180
119591140,0ptp4l0-21swapper/009:10:220
119591140,0ptp4l0-21swapper/008:49:330
119591140,0ptp4l0-21swapper/008:35:220
119591140,0ptp4l0-21swapper/008:15:210
119591140,0ptp4l0-21swapper/007:46:380
119591140,0ptp4l0-21swapper/007:30:210
119591140,0ptp4l0-21swapper/007:15:230
16173991311,1cyclictest33-21ksoftirqd/212:10:232
119591130,0ptp4l0-21swapper/311:55:153
119591130,0ptp4l0-21swapper/211:00:142
119591130,0ptp4l0-21swapper/210:55:142
119591130,0ptp4l0-21swapper/210:40:242
119591130,0ptp4l0-21swapper/210:30:202
119591130,0ptp4l0-21swapper/210:25:142
119591130,0ptp4l0-21swapper/210:10:132
119591130,0ptp4l0-21swapper/210:05:162
119591130,0ptp4l0-21swapper/210:00:132
119591130,0ptp4l0-21swapper/209:56:302
119591130,0ptp4l0-21swapper/209:45:212
119591130,0ptp4l0-21swapper/209:30:162
119591130,0ptp4l0-21swapper/209:05:162
119591130,0ptp4l0-21swapper/209:00:182
119591130,0ptp4l0-21swapper/208:30:132
119591130,0ptp4l0-21swapper/208:20:202
119591130,0ptp4l0-21swapper/207:55:182
119591130,0ptp4l0-21swapper/207:50:202
119591130,0ptp4l0-21swapper/207:45:142
119591130,0ptp4l0-21swapper/207:40:162
119591130,0ptp4l0-21swapper/207:35:122
119591130,0ptp4l0-21swapper/207:10:182
119591130,0ptp4l0-21swapper/112:15:161
119591130,0ptp4l0-21swapper/111:55:121
119591130,0ptp4l0-21swapper/111:45:151
119591130,0ptp4l0-21swapper/111:35:131
119591130,0ptp4l0-21swapper/111:30:021
119591130,0ptp4l0-21swapper/111:20:141
119591130,0ptp4l0-21swapper/111:00:121
119591130,0ptp4l0-21swapper/110:55:181
119591130,0ptp4l0-21swapper/110:35:121
119591130,0ptp4l0-21swapper/110:30:121
119591130,0ptp4l0-21swapper/110:10:131
119591130,0ptp4l0-21swapper/110:05:121
119591130,0ptp4l0-21swapper/109:45:131
119591130,0ptp4l0-21swapper/109:35:141
119591130,0ptp4l0-21swapper/109:30:111
119591130,0ptp4l0-21swapper/109:20:141
119591130,0ptp4l0-21swapper/109:15:131
119591130,0ptp4l0-21swapper/109:05:131
119591130,0ptp4l0-21swapper/107:55:111
119591130,0ptp4l0-21swapper/107:45:121
119591130,0ptp4l0-21swapper/012:10:160
119591130,0ptp4l0-21swapper/012:05:140
119591130,0ptp4l0-21swapper/011:50:150
119591130,0ptp4l0-21swapper/011:30:140
119591130,0ptp4l0-21swapper/011:25:150
119591130,0ptp4l0-21swapper/011:20:120
119591130,0ptp4l0-21swapper/011:10:150
119591130,0ptp4l0-21swapper/010:45:130
119591130,0ptp4l0-21swapper/010:40:210
119591130,0ptp4l0-21swapper/010:00:140
119591130,0ptp4l0-21swapper/009:55:140
119591130,0ptp4l0-21swapper/009:40:160
119591130,0ptp4l0-21swapper/009:15:120
119591130,0ptp4l0-21swapper/008:50:130
119591130,0ptp4l0-21swapper/008:05:120
119591130,0ptp4l0-21swapper/008:00:140
119591130,0ptp4l0-21swapper/007:40:110
119591130,0ptp4l0-21swapper/007:10:140
119591120,0ptp4l0-21swapper/211:51:222
119591120,0ptp4l0-21swapper/209:15:192
119591120,0ptp4l0-21swapper/208:50:242
119591110,0ptp4l0-21swapper/209:35:182
119591100,1ptp4l0-21swapper/207:27:422
119591100,0ptp4l0-21swapper/309:41:063
119591100,0ptp4l0-21swapper/211:32:342
119591100,0ptp4l0-21swapper/209:13:232
119591100,0ptp4l0-21swapper/208:48:322
119591100,0ptp4l0-21swapper/207:30:302
161749997,1cyclictest28960-21tr11:35:143
161749997,1cyclictest28926-21cut07:30:173
161749997,1cyclictest26992sleep308:45:043
161749997,1cyclictest22492-21taskset12:23:573
161749997,1cyclictest21754-21cat10:20:163
161749997,1cyclictest17060-21sendmail_mailst11:10:213
161749997,1cyclictest1449-21taskset10:44:073
161749997,1cyclictest14392-21chrt12:07:173
161749997,1cyclictest10859-21cut12:00:193
161749995,1cyclictest20039-21sort12:20:013
161749990,8cyclictest30844-21sed08:35:163
11959190,0ptp4l0-21swapper/311:28:463
11959190,0ptp4l0-21swapper/310:05:013
11959190,0ptp4l0-21swapper/212:35:362
11959190,0ptp4l0-21swapper/212:32:162
11959190,0ptp4l0-21swapper/212:26:512
11959190,0ptp4l0-21swapper/212:17:102
11959190,0ptp4l0-21swapper/212:06:092
11959190,0ptp4l0-21swapper/212:00:332
11959190,0ptp4l0-21swapper/211:56:162
11959190,0ptp4l0-21swapper/211:45:342
11959190,0ptp4l0-21swapper/211:20:502
11959190,0ptp4l0-21swapper/211:16:112
11959190,0ptp4l0-21swapper/211:10:542
11959190,0ptp4l0-21swapper/211:05:242
11959190,0ptp4l0-21swapper/210:51:552
11959190,0ptp4l0-21swapper/210:46:522
11959190,0ptp4l0-21swapper/210:20:402
11959190,0ptp4l0-21swapper/210:15:022
11959190,0ptp4l0-21swapper/209:50:272
11959190,0ptp4l0-21swapper/209:43:182
11959190,0ptp4l0-21swapper/208:56:042
11959190,0ptp4l0-21swapper/208:44:392
11959190,0ptp4l0-21swapper/208:37:562
11959190,0ptp4l0-21swapper/208:28:082
11959190,0ptp4l0-21swapper/208:13:302
11959190,0ptp4l0-21swapper/208:05:082
11959190,0ptp4l0-21swapper/208:00:062
11959190,0ptp4l0-21swapper/207:21:432
11959190,0ptp4l0-21swapper/207:18:302
161749986,1cyclictest93742sleep310:56:153
161749986,1cyclictest8674-21cut07:55:103
161749986,1cyclictest8583-21cat08:55:143
161749986,1cyclictest804-21cat11:40:233
161749986,1cyclictest8011-21expr09:55:133
161749986,1cyclictest6751-21ntpq08:50:193
161749986,1cyclictest622-21chrt07:40:113
161749986,1cyclictest5961-21basename07:50:123
161749986,1cyclictest4983-21cat10:50:153
161749986,1cyclictest4975-21taskset09:50:083
161749986,1cyclictest3996-21chrt10:47:503
161749986,1cyclictest3609-21cut07:45:133
161749986,1cyclictest31844-21open_inodes07:35:173
161749986,1cyclictest29827-21expr09:35:133
161749986,1cyclictest29234-21uname12:35:173
161749986,1cyclictest28850-21basename10:35:113
161749986,1cyclictest28249-21sed08:30:163
161749986,1cyclictest27190-21expr09:30:133
161749986,1cyclictest26809-21grep10:30:143
161749986,1cyclictest26369-21expr11:30:123
161749986,1cyclictest2559-21cat09:45:143
161749986,1cyclictest25575-21cat07:25:143
161749986,1cyclictest25305-21cat08:25:153
161749986,1cyclictest24497-21expr09:25:143
161749986,1cyclictest24155-21grep10:25:143
161749986,1cyclictest23982-21chrt08:21:023
161749986,1cyclictest23719-21sed12:25:143
161749986,1cyclictest23109-21gzip09:20:203
161749986,1cyclictest22467-21tr07:20:163
161749986,1cyclictest209402sleep309:16:073
161749986,1cyclictest2084-21df11:45:153
161749986,1cyclictest20421-21cat07:15:173
161749986,1cyclictest20405-21taskset10:16:473
161749986,1cyclictest19853-21tail08:15:153
161749986,1cyclictest18498-21chrt09:13:423
161749986,1cyclictest17647-21cut07:10:203
161749986,1cyclictest16849-21tr08:10:133
161749986,1cyclictest16814-21cat10:10:183
161749986,1cyclictest159022sleep309:09:303
161749986,1cyclictest15325-21sh12:10:143
161749986,1cyclictest14957-21taskset11:06:583
161749986,1cyclictest14422-21forks08:05:133
161749986,1cyclictest1388-21cut08:40:163
161749986,1cyclictest12209-21tail09:00:173
161749986,1cyclictest12008-21basename08:00:163
161739986,1cyclictest29159-21cat11:35:142
161739986,1cyclictest21238-21head08:15:232
11959180,0ptp4l0-21swapper/212:20:402
11959180,0ptp4l0-21swapper/211:25:032
161749975,1cyclictest5427-21latency11:50:163
161749975,1cyclictest21884-21uname11:20:163
161749975,1cyclictest12450-21chrt10:01:433
161749975,1cyclictest10631-21hddtemp_smartct11:00:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional