You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-02 - 10:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Wed Sep 02, 2026 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26217995022,17cyclictest35966irq/58-eth021:50:133
2619199472,32cyclictest11892-21diskmemload00:25:140
2620799462,34cyclictest5796-21ssh23:30:142
2619999441,36cyclictest18509-21ssh23:55:121
2619199441,33cyclictest25445-21ssh23:05:140
26207994215,12cyclictest12718-21ssh22:40:122
2619199423,29cyclictest12714-21ssh22:40:120
2619199422,34cyclictest30763-21ssh22:50:130
2621799411,30cyclictest35966irq/58-eth022:20:143
26199994125,9cyclictest13170-21apt-get19:30:141
26199994125,9cyclictest13170-21apt-get19:30:131
2620799396,14cyclictest0-21swapper/200:15:132
2620799392,28cyclictest261rcuc/221:40:132
2619199394,24cyclictest18005-21rm22:25:140
2619199390,29cyclictest16839-21ssh00:30:130
2619999381,26cyclictest19701-21ssh21:50:131
2620799376,28cyclictest25771-21ssh00:35:132
26207993712,9cyclictest27-21ksoftirqd/223:20:132
2620799371,27cyclictest21700-21ssh22:45:142
2619199377,20cyclictest12-21ksoftirqd/000:10:140
2619199375,24cyclictest7380-21ssh22:55:120
2619199372,28cyclictest23915-21ssh23:40:140
26217993625,7cyclictest35966irq/58-eth021:15:173
26191993626,8cyclictest0-21swapper/022:15:130
2619199362,18cyclictest11540-21usage_EthcTask21:25:290
26199993519,12cyclictest12266-21apt-get21:10:121
2621799341,19cyclictest35966irq/58-eth000:35:123
2620799336,23cyclictest0-21swapper/223:55:122
2619199332,25cyclictest9502-21ssh23:50:140
2620799324,19cyclictest967-1kworker/u9:221:50:132
26199993224,4cyclictest26345-21apt-get19:10:131
2621799315,21cyclictest35966irq/58-eth023:30:143
2621799313,25cyclictest0-21swapper/300:30:153
2621799313,13cyclictest35966irq/58-eth023:25:123
2621799311,6cyclictest0-21swapper/319:10:153
2620799314,20cyclictest0-21swapper/200:30:152
2620799314,16cyclictest27346irq/40-dwc2_hso20:50:132
2620799313,15cyclictest0-21swapper/220:00:122
26199993120,8cyclictest22845-21apt-get19:40:131
2621799302,23cyclictest35966irq/58-eth021:20:133
2620799303,18cyclictest27346irq/40-dwc2_hso19:25:122
2620799303,16cyclictest261rcuc/223:50:122
2620799302,26cyclictest3992-21ssh22:51:292
2619999303,14cyclictest3698-21ssh22:35:131
26199993022,4cyclictest28793-21apt-get20:20:131
2619199302,22cyclictest14860-21ssh23:35:140
2619999294,18cyclictest25033-21ssh21:35:131
26199992923,4cyclictest16431-21apt-get23:00:111
26199992923,4cyclictest16431-21apt-get23:00:101
2619999292,20cyclictest8996-21ssh22:20:141
26199992913,8cyclictest0-21swapper/119:20:131
2620799283,20cyclictest0-21swapper/219:20:132
2620799283,16cyclictest27346irq/40-dwc2_hso20:30:142
2620799282,20cyclictest0-21swapper/220:25:152
2620799282,14cyclictest1933-21ssh22:33:292
2620799282,11cyclictest0-21swapper/222:15:152
2620799281,19cyclictest14372-21apt-get23:35:032
2620799281,13cyclictest93066snmpd20:10:312
2619999287,5cyclictest21-21ksoftirqd/122:26:191
2619999283,18cyclictest4153-21ssh00:05:141
2619999282,21cyclictest28815-21ssh21:55:151
26199992816,8cyclictest9478-21apt-get23:50:131
2619999281,5cyclictest0-21swapper/123:30:131
26199992815,9cyclictest12706-21apt-get22:40:121
2619199282,21cyclictest22166-21ssh00:15:130
2619199282,20cyclictest15887-21ssh21:10:240
2621799274,19cyclictest35966irq/58-eth020:20:143
2621799273,18cyclictest0-21swapper/322:33:293
2621799271,12cyclictest35966irq/58-eth000:05:133
2620799272,20cyclictest0-21swapper/219:45:132
2620799271,13cyclictest16439-21ssh23:00:112
2620799271,13cyclictest16439-21ssh23:00:112
2619999275,18cyclictest29135-21ssh23:25:131
2619999272,19cyclictest14418-21ssh21:45:291
2619999272,19cyclictest14418-21ssh21:45:291
26199992719,4cyclictest30006-21apt-get20:55:131
26199992719,4cyclictest1614-21apt-get21:40:131
26199992718,6cyclictest22160-21apt-get00:15:151
26199992713,11cyclictest0-21swapper/120:05:111
2619199275,15cyclictest5780-21ssh23:30:130
2621799261,6cyclictest0-21swapper/323:15:213
2621799261,6cyclictest0-21swapper/321:00:183
26217992615,8cyclictest35966irq/58-eth022:25:143
2621799261,16cyclictest35966irq/58-eth022:40:123
2620799263,15cyclictest27346irq/40-dwc2_hso20:15:132
2620799262,19cyclictest0-21swapper/222:20:142
26207992610,11cyclictest0-21swapper/221:45:132
26207992610,11cyclictest0-21swapper/221:45:132
2619999262,22cyclictest1886-21sed21:20:271
26199992614,8cyclictest14444-21apt-get22:05:131
2621799254,17cyclictest35966irq/58-eth021:10:113
2621799252,15cyclictest0-21swapper/323:05:143
2621799251,7cyclictest0-21swapper/321:55:283
2621799251,6cyclictest0-21swapper/321:30:403
2621799251,6cyclictest0-21swapper/320:45:153
2620799253,8cyclictest82311kthreadcore21:05:182
2620799252,20cyclictest0-21swapper/222:35:142
2620799252,19cyclictest0-21swapper/220:05:132
2620799252,18cyclictest13496-21ssh21:27:292
26207992515,5cyclictest1675-21ssh23:10:012
2620799251,15cyclictest0-21swapper/221:00:142
2620799251,11cyclictest202666ssh23:10:132
2620799251,10cyclictest23446-21ssh22:10:142
2619999256,14cyclictest0-21swapper/119:25:121
2619999252,16cyclictest5495-21ssh22:00:151
2619999251,18cyclictest13192-21ssh00:10:151
2619999251,17cyclictest23450-21ssh22:10:151
2621799248,4cyclictest0-21swapper/323:22:143
2621799248,4cyclictest0-21swapper/321:40:333
2621799248,11cyclictest0-21swapper/321:37:153
2621799246,13cyclictest0-21swapper/300:10:143
2621799245,6cyclictest0-21swapper/300:29:403
2621799241,8cyclictest0-21swapper/322:15:143
2621799241,6cyclictest0-21swapper/323:40:183
2621799241,6cyclictest0-21swapper/322:50:163
2621799241,6cyclictest0-21swapper/320:40:203
2621799241,6cyclictest0-21swapper/319:25:013
2621799241,20cyclictest0-21swapper/323:50:233
2620799242,16cyclictest27346irq/40-dwc2_hso23:45:132
2620799242,15cyclictest7816-21apt-get00:25:122
2620799242,12cyclictest22890-21users22:25:332
26207992413,9cyclictest30067-21ssh21:35:442
26207992413,9cyclictest13297-21ssh22:04:082
2620799240,10cyclictest1881-21grep19:50:212
2619999242,17cyclictest29048-21ssh22:48:281
26199992417,4cyclictest25216-21apt-get20:50:121
26199992415,6cyclictest7368-21apt-get22:55:131
2619199249,8cyclictest0-21swapper/019:40:120
2619199242,19cyclictest21013-21ssh21:30:440
2619199242,18cyclictest14405-21ntp_states21:45:290
2619199242,18cyclictest14405-21ntp_states21:45:290
2619199242,17cyclictest14462-21sh22:05:130
2619199242,15cyclictest21366-21ssh21:15:170
2621799238,4cyclictest0-21swapper/322:11:053
2621799238,4cyclictest0-21swapper/321:26:353
2621799231,9cyclictest0-21swapper/323:00:163
2621799231,9cyclictest0-21swapper/323:00:163
2621799231,7cyclictest0-21swapper/322:55:213
2621799231,7cyclictest0-21swapper/300:15:233
2621799231,6cyclictest0-21swapper/323:45:173
2621799231,6cyclictest0-21swapper/323:10:183
2621799231,6cyclictest0-21swapper/322:45:183
2621799231,6cyclictest0-21swapper/322:35:283
2621799231,6cyclictest0-21swapper/322:05:013
2621799231,6cyclictest0-21swapper/320:35:163
2621799231,6cyclictest0-21swapper/320:15:283
2621799231,6cyclictest0-21swapper/320:00:173
2621799231,6cyclictest0-21swapper/319:35:163
2621799231,6cyclictest0-21swapper/300:25:013
2621799231,6cyclictest0-21swapper/300:00:193
2621799231,5cyclictest0-21swapper/319:15:293
26217992314,6cyclictest35966irq/58-eth022:05:143
26217992310,10cyclictest35966irq/58-eth021:45:143
26217992310,10cyclictest35966irq/58-eth021:45:143
2620799239,11cyclictest13639-21kthreadcore23:15:222
2620799234,15cyclictest32366-21kthreadcore19:15:202
2620799232,17cyclictest27346irq/40-dwc2_hso21:30:132
2620799232,16cyclictest0-21swapper/221:22:282
26207992311,5cyclictest30260-21ssh23:42:162
2620799231,10cyclictest0-21swapper/222:55:112
2619999237,3cyclictest0-21swapper/123:11:111
2619999237,12cyclictest0-21swapper/120:15:121
2619999234,4cyclictest21-21ksoftirqd/121:15:311
2619999234,4cyclictest201rcuc/100:22:201
2619999234,15cyclictest0-21swapper/121:25:111
2619999233,7cyclictest21220-21ssh23:37:171
2619999233,13cyclictest0-21swapper/120:35:131
26199992314,7cyclictest7819-21ssh00:25:121
26199992313,7cyclictest0-21swapper/100:35:131
2619999230,10cyclictest29044-21rm23:40:481
2619199239,11cyclictest32209-21cron19:50:010
2619199231,17cyclictest1631-21ssh21:40:130
2621799221,6cyclictest0-21swapper/323:55:313
2621799221,6cyclictest0-21swapper/323:35:143
2621799221,6cyclictest0-21swapper/321:05:193
2621799221,6cyclictest0-21swapper/320:55:223
2621799221,6cyclictest0-21swapper/320:35:013
2621799221,6cyclictest0-21swapper/320:10:183
2621799221,6cyclictest0-21swapper/319:40:213
2621799221,6cyclictest0-21swapper/319:30:233
2621799221,6cyclictest0-21swapper/319:30:233
2621799221,6cyclictest0-21swapper/319:25:153
2621799221,5cyclictest0-21swapper/320:05:253
2620799228,7cyclictest21981-21cat00:15:022
2620799223,14cyclictest27346irq/40-dwc2_hso21:15:152
2620799222,11cyclictest2404-21latency_hist21:00:012
2620799221,17cyclictest29370-21cut00:00:222
2620799221,12cyclictest31201-21ssh00:20:122
2619999228,11cyclictest0-21swapper/122:30:131
2619999224,5cyclictest21-21ksoftirqd/123:23:191
26199992213,6cyclictest15986-21ssh21:30:121
26191992210,3cyclictest131rcu_preempt23:20:010
26191992210,3cyclictest131rcu_preempt21:50:260
2621799211,6cyclictest0-21swapper/320:50:173
2621799211,6cyclictest0-21swapper/320:25:223
2621799211,6cyclictest0-21swapper/319:50:213
2621799211,6cyclictest0-21swapper/319:45:203
2621799211,5cyclictest0-21swapper/319:55:243
2620799217,11cyclictest0-21swapper/222:05:202
2620799213,15cyclictest13172-21kthreadcore20:35:202
2620799212,15cyclictest5090-21awk22:00:022
2620799211,9cyclictest14452-21ssh21:10:202
26207992115,3cyclictest22512-21idleruntime-cro19:40:012
2620799211,10cyclictest29083-21find23:25:112
26207992110,8cyclictest0-21swapper/219:55:322
26207992110,4cyclictest13505-21tr19:30:192
26207992110,4cyclictest13505-21tr19:30:192
2619999216,4cyclictest0-21swapper/123:07:491
2619999216,12cyclictest0-21swapper/120:10:141
2619999215,10cyclictest0-21swapper/120:00:121
2619999214,8cyclictest21-21ksoftirqd/123:19:021
2619999214,5cyclictest21-21ksoftirqd/122:50:211
2619999214,5cyclictest21-21ksoftirqd/100:03:141
2619999212,3cyclictest21-21ksoftirqd/122:15:171
26199992116,2cyclictest456-21apt-get23:45:141
2619199213,10cyclictest16429-21ssh23:00:100
2619199213,10cyclictest16429-21ssh23:00:100
2619199211,15cyclictest6807-21latency19:20:230
26191992110,3cyclictest131rcu_preempt00:35:280
2620799206,10cyclictest0-21swapper/219:40:142
2620799205,13cyclictest4128-21sh00:05:122
2620799201,15cyclictest26236-21latency_hist19:10:012
2620799201,10cyclictest21370-21kthreadcore20:45:202
2620799201,10cyclictest17296-21kthreadcore20:40:212
2619999207,10cyclictest20474-21diskstats20:45:171
2619999204,4cyclictest0-21swapper/119:35:291
2619999203,4cyclictest16860-21cpu00:30:151
2619999201,11cyclictest0-21swapper/119:45:111
2619199208,7cyclictest0-21swapper/023:45:140
2619199203,11cyclictest11892-21diskmemload00:00:150
2619199202,12cyclictest4149-21ssh00:05:140
2619199201,8cyclictest27434-21irqcore22:30:190
26191992012,5cyclictest21666-21tr23:55:220
2619199201,14cyclictest28805-21ssh21:55:140
2620799191,7cyclictest0-21swapper/220:20:132
2620799191,15cyclictest28657-21kthreadcore19:10:222
2619999195,4cyclictest0-21swapper/120:25:251
2619199199,7cyclictest2185-21cron21:00:010
2619199199,7cyclictest1830-21threads21:20:280
2619199197,9cyclictest28787-21cat20:50:270
2619199192,12cyclictest3226-21apt-get22:35:030
2619199192,11cyclictest0-21swapper/019:10:150
2619199191,15cyclictest4806-21apt-get19:55:150
2619199191,14cyclictest20565-21uniq20:45:170
2619199191,13cyclictest5480-21ssh22:00:140
2619999188,4cyclictest0-21swapper/119:50:131
2619999187,8cyclictest31968-21cut19:15:201
2619999181,5cyclictest21-21ksoftirqd/119:55:251
2619999181,14cyclictest7494-21uniq21:05:161
2619999181,11cyclictest2671-21ls21:00:161
2619199189,7cyclictest3424-21ntpq19:50:250
2619199189,7cyclictest2112-21df23:10:150
2619199181,8cyclictest5891-21latency_hist20:30:020
2619199181,7cyclictest9081-21df22:20:150
2619199181,14cyclictest5313-21latency_hist23:30:020
26191991811,4cyclictest25230-21missed_timers22:45:250
2619199181,13cyclictest32453-21tr00:20:200
2619199181,13cyclictest11892-21diskmemload22:10:270
26191991810,6cyclictest28644-21cron23:25:010
26191991810,6cyclictest2572-21cut21:00:160
2619999178,6cyclictest0-21swapper/120:40:121
2619999170,11cyclictest6172-21ls20:30:181
2619199179,5cyclictest8175-21kthreadcore21:05:190
2619199173,11cyclictest6572-21irqrtprio20:30:200
2619199173,11cyclictest15271-21tr19:30:220
2619199173,11cyclictest15271-21tr19:30:220
2619199172,12cyclictest26323-21kthreadcore20:15:210
2619199172,11cyclictest8319-21acpi19:25:100
2619199171,7cyclictest18521-21irqrtprio19:35:180
2619199171,13cyclictest19429-21idleruntime20:10:180
2619199171,11cyclictest12-21ksoftirqd/020:05:020
2619199165,8cyclictest25125-21df_inode21:35:140
2619199162,11cyclictest31796-21kthreadcore19:15:200
2619199162,11cyclictest20154-21munin-run20:45:010
2619199162,11cyclictest18983-21latency_hist20:10:010
2619199162,10cyclictest0-21swapper/020:35:130
2619199152,9cyclictest0-21swapper/020:20:140
2619999146,5cyclictest26274-21latency_hist19:10:011
2619199149,3cyclictest26257-21ls19:10:010
2621799111,7cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional