You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-01 - 10:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Thu Jan 01, 2026 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2902729380,9sleep00-21swapper/019:06:210
2897128842,41sleep10-21swapper/119:05:371
2929828674,8sleep20-21swapper/219:09:522
2924728169,8sleep30-21swapper/319:09:133
2940099231,20cyclictest0-21swapper/221:15:152
2940399212,16cyclictest0-21swapper/300:00:153
2940399208,11cyclictest0-21swapper/323:30:143
29403992015,4cyclictest0-21swapper/322:30:203
29393992019,1cyclictest0-21swapper/020:40:130
29403991914,4cyclictest0-21swapper/320:10:013
2940099190,1cyclictest0-21swapper/222:55:142
2939399190,16cyclictest0-21swapper/022:15:130
2940399177,6cyclictest22105-21cron21:09:593
29403991712,3cyclictest31055-21cut22:35:133
29403991710,3cyclictest0-21swapper/320:25:143
29403991614,1cyclictest0-21swapper/319:50:193
29403991613,1cyclictest0-21swapper/300:35:143
29403991611,3cyclictest20775-21cut23:20:163
29393991613,2cyclictest17421-21cron00:25:000
2940399159,4cyclictest12991-21timerandwakeup19:40:393
29403991513,1cyclictest0-21swapper/322:45:193
29400991511,3cyclictest4780-21grep20:30:262
29400991511,2cyclictest15990-21grep19:50:002
2939899150,10cyclictest0-21swapper/119:20:141
2939399156,8cyclictest30336-21awk20:20:000
2939399152,12cyclictest10334-21apt-get00:10:100
29393991515,0cyclictest0-21swapper/000:34:430
2940399147,2cyclictest537-21snmpd20:50:123
29403991412,1cyclictest0-21swapper/322:05:143
29403991411,2cyclictest1273-21grep19:15:243
29403991410,3cyclictest5867-21tr21:40:213
29403991410,2cyclictest19435-21grep23:15:283
2940399140,12cyclictest0-21swapper/319:25:143
29400991410,3cyclictest28460-21wc21:20:212
29400991410,3cyclictest12501-21wc19:40:312
2940099140,0cyclictest0-21swapper/219:10:132
29398991410,3cyclictest16085-21cat19:50:001
2939399141,6cyclictest19822-21perf00:29:590
29393991410,3cyclictest29181-21grep22:30:170
2940399137,4cyclictest29721-21taskset23:40:103
2940399133,4cyclictest19947-21kernelversion22:10:203
2940399132,7cyclictest0-21swapper/300:30:153
29403991311,1cyclictest0-21swapper/322:40:133
29403991311,1cyclictest0-21swapper/300:25:143
29403991310,2cyclictest10225-21dump-pmu-power20:44:593
29400991310,2cyclictest32705-21munin-plugin-st20:25:012
29400991310,2cyclictest25602-21wc20:10:012
29400991310,2cyclictest23122-21sh20:05:002
29400991310,2cyclictest18534-21grep23:15:192
29400991310,2cyclictest13464-21sort20:50:162
2940099130,1cyclictest0-21swapper/223:00:142
2940099130,1cyclictest0-21swapper/222:15:142
2940099130,12cyclictest5552-21grep23:55:182
29398991310,2cyclictest24009-21cat00:35:281
29398991310,2cyclictest18816-21tr00:25:211
29398991310,2cyclictest15471-21ls22:00:211
29398991310,2cyclictest14934-21basename23:09:591
2939399139,3cyclictest20813-21cat20:00:010
2939399136,3cyclictest14815-21cron00:20:010
29393991312,1cyclictest0-21swapper/022:20:140
29393991310,2cyclictest8655-21grep22:55:160
29393991310,2cyclictest5937-21ls21:40:230
29393991310,2cyclictest30868-21tr20:20:160
29393991310,2cyclictest27139-21wc23:35:010
29393991310,2cyclictest26997-21grep20:10:250
29393991310,2cyclictest19925-21wc21:05:010
29393991310,2cyclictest15041-21wc20:55:000
29393991310,2cyclictest13092-21tr23:05:200
2939399130,10cyclictest0-21swapper/023:25:140
2940399129,2cyclictest9146-21tr22:55:203
2940399129,2cyclictest7857-21sort20:39:593
2940399129,2cyclictest4754-21sed20:30:263
2940399129,2cyclictest4458-21idleruntime-cro21:40:013
2940399129,2cyclictest30642-21cut21:25:143
2940399129,2cyclictest22581-21tr21:10:153
2940399129,2cyclictest18846-21tr19:55:143
2940399129,2cyclictest15904-21cut20:55:183
2940399129,2cyclictest15019-21cut22:00:183
2940399129,2cyclictest13492-21sed00:15:183
2940399129,2cyclictest11224-21tail20:45:193
2940399129,2cyclictest11176-21head21:50:243
2940399129,2cyclictest0-21swapper/322:20:143
2940399128,3cyclictest31730-21sh19:15:003
2940399127,2cyclictest225-21systemd-journal23:39:043
2940399125,3cyclictest225-21systemd-journal23:15:003
2940399122,9cyclictest11097-21taskset19:38:413
2940399122,3cyclictest6067-21sed23:55:253
2940399121,10cyclictest0-21swapper/319:20:153
2940099129,2cyclictest8608-21grep20:40:152
2940099129,2cyclictest8217-21grep21:45:192
2940099129,2cyclictest6573-21cat19:30:012
2940099129,2cyclictest32440-21cat22:35:242
2940099129,2cyclictest32387-21cut23:45:142
2940099129,2cyclictest27869-21munin-plugin-st20:15:002
2940099129,2cyclictest23997-21grep22:20:152
2940099129,2cyclictest20741-21grep21:05:172
2940099129,2cyclictest20326-21tr00:30:142
2940099129,2cyclictest17858-21grep21:00:162
2940099129,2cyclictest16701-21cut22:05:142
2940099129,2cyclictest12441-21sh00:15:002
2940099128,3cyclictest24959-21grep23:30:152
29400991210,1cyclictest15518-21grep20:55:162
2939899129,2cyclictest8239-21cat19:30:251
2939899129,2cyclictest6188-21grep20:35:191
2939899129,2cyclictest610-21tr20:25:151
2939899129,2cyclictest26795-21grep22:25:151
2939899129,2cyclictest26579-21grep23:30:281
2939899129,2cyclictest22272-21cron23:24:591
2939899129,2cyclictest2162-21grep22:40:221
2939899129,2cyclictest20723-21head22:10:271
2939899129,2cyclictest14816-21sh00:20:011
2939899129,2cyclictest10230-21tr19:35:221
2939899128,3cyclictest628-21cron22:40:001
2939899128,3cyclictest12236-21expr21:55:161
2939899123,6cyclictest28588-21apt-get22:30:151
29398991210,1cyclictest7346-21grep00:00:161
2939899120,8cyclictest0-21swapper/121:00:141
2939899120,11cyclictest6459-21grep22:50:161
2939399129,2cyclictest9143-21cut19:35:160
2939399129,2cyclictest5546-21grep20:35:160
2939399129,2cyclictest32287-21tr19:15:140
2939399129,2cyclictest27521-21grep23:35:130
2939399129,2cyclictest24483-21cron21:14:590
2939399129,2cyclictest2119-21grep23:50:160
2939399129,2cyclictest2119-21grep23:50:150
2939399129,2cyclictest14855-21grep22:00:160
2939399129,2cyclictest11204-21grep20:45:190
2939399129,1cyclictest0-21swapper/023:00:140
2939399128,3cyclictest27660-21tr21:20:160
2939399128,3cyclictest20954-21head23:20:160
2939399126,2cyclictest225-21systemd-journal00:00:000
2940399118,2cyclictest9922-21chrt00:08:193
2940399118,2cyclictest8429-21sed19:30:263
2940399118,2cyclictest5741-21cut22:50:133
2940399118,2cyclictest32506-21taskset20:22:363
2940399118,2cyclictest27632-21cut21:20:163
2940399118,2cyclictest26910-21cut22:25:153
2940399118,2cyclictest15943-21cut00:20:163
2940399118,2cyclictest13742-21chrt21:57:123
2940399118,2cyclictest13624-21sh23:05:233
2940399117,3cyclictest11078-21cut00:10:153
2940399115,3cyclictest225-21systemd-journal23:55:013
2940399115,3cyclictest225-21systemd-journal23:55:003
2940399115,2cyclictest225-21systemd-journal20:15:003
2940399114,3cyclictest225-21systemd-journal21:46:313
2940399114,3cyclictest225-21systemd-journal21:17:333
2940399110,9cyclictest15995-21perf19:50:003
2940399110,10cyclictest29791-21cat20:15:273
2940099119,1cyclictest12284-21chrt20:46:082
2940099118,2cyclictest9683-21rsyslog00:05:342
2940099118,2cyclictest7469-21chrt20:38:152
2940099118,2cyclictest614-21cut21:30:172
2940099118,2cyclictest31687-21taskset23:42:542
2940099118,2cyclictest30287-21grep21:25:122
2940099118,2cyclictest2575-21tail22:40:262
2940099118,2cyclictest24755-21wc00:39:592
2940099118,2cyclictest20823-21grep23:20:162
2940099118,2cyclictest20137-21tr22:10:212
2940099118,2cyclictest19742-21chrt00:29:262
2940099118,2cyclictest17895-21tail19:50:252
2940099118,2cyclictest17505-21cron23:15:002
2940099118,2cyclictest12607-21grep21:55:172
2940099113,6cyclictest3315-21apt-get22:45:152
2940099113,6cyclictest20746-21awk20:00:002
2940099113,5cyclictest4457-21cron21:40:012
2940099110,8cyclictest0-21swapper/220:25:142
2939899119,1cyclictest4456-21sh21:40:001
2939899119,1cyclictest20318-21sed19:55:271
2939899118,2cyclictest9162-21cut22:55:191
2939899118,2cyclictest8873-21sed20:40:171
2939899118,2cyclictest870-21processes19:15:211
2939899118,2cyclictest6199-21chrt19:26:421
2939899118,2cyclictest3401-21grep20:30:151
2939899118,2cyclictest30975-21grep23:40:241
2939899118,2cyclictest30583-21basename20:20:131
2939899118,2cyclictest29929-21cat19:10:151
2939899118,2cyclictest25486-21cron20:10:011
2939899118,2cyclictest23130-21grep21:10:171
2939899118,2cyclictest20794-21chrt21:05:181
2939899118,2cyclictest16895-21tr22:05:151
2939899118,2cyclictest16852-21grep20:55:261
2939899118,2cyclictest16269-21cut00:20:191
2939899118,2cyclictest12236-21grep19:40:291
2939899118,2cyclictest11686-21grep23:00:231
2939899118,2cyclictest1167-21sed21:30:221
2939899117,3cyclictest9724-21taskset00:05:431
2939899117,1cyclictest0-21swapper/123:15:161
2939399119,1cyclictest18516-21grep23:15:190
2939399118,2cyclictest8369-21awk00:00:250
2939399118,2cyclictest6877-21ls22:50:220
2939399118,2cyclictest31732-21sh19:15:000
2939399118,2cyclictest31534-21df22:35:160
2939399118,2cyclictest2997-21grep21:35:170
2939399118,2cyclictest23209-21grep00:35:180
2939399118,2cyclictest20653-21grep21:05:170
2939399118,2cyclictest16859-21tr22:05:150
2939399118,2cyclictest16504-21expr19:50:120
2939399118,2cyclictest10595-21uname21:50:170
2939399118,2cyclictest10080-21turbostat00:10:010
2939399117,3cyclictest8831-21cron19:35:000
2939399117,3cyclictest712-21grep21:30:180
2939399117,2cyclictest20145-21tr22:10:220
2939399115,2cyclictest225-21systemd-journal21:50:000
113362110,1sleep10-21swapper/121:51:481
2940399108,1cyclictest19628-21taskset21:03:433
2940399107,2cyclictest21092-21cut20:00:153
2940399107,2cyclictest1811-21chrt21:31:363
2940399107,2cyclictest1802-21taskset23:49:573
2940399107,2cyclictest11576-21awk23:00:223
2940399107,2cyclictest0-21swapper/323:25:263
2940399107,1cyclictest0-21swapper/322:15:143
2940099108,1cyclictest6428-21grep21:40:282
2940099108,1cyclictest3398-21taskset23:50:252
2940099108,1cyclictest3398-21taskset23:50:252
2940099108,1cyclictest1942-21chrt19:20:012
2940099108,1cyclictest11261-21taskset21:50:502
2940099107,2cyclictest7463-21tail19:30:162
2940099107,2cyclictest7378-21grep22:50:262
2940099107,2cyclictest28685-21wget22:30:152
2940099107,2cyclictest23930-21cut23:25:242
2940099107,2cyclictest15221-21dpkg00:20:012
2940099107,2cyclictest13061-21grep23:05:202
2940099107,2cyclictest1231-21tail19:15:242
2940099100,9cyclictest9119-21cut19:35:172
2940099100,9cyclictest29516-21grep20:15:232
2940099100,9cyclictest273492sleep223:35:122
2940099100,9cyclictest13718-21sed00:15:202
2939899108,2cyclictest24254-21awk23:25:271
2939899108,2cyclictest21374-21expr20:00:161
2939899108,1cyclictest38150irq/131-snd_hda21:20:161
2939899108,1cyclictest28664-21cat20:15:151
2939899108,1cyclictest25584-21chrt22:21:201
2939899108,1cyclictest2085-21apache_volume23:50:151
2939899108,1cyclictest2085-21apache_volume23:50:151
2939899108,1cyclictest18300-21taskset19:54:321
2939899107,2cyclictest31797-21taskset21:25:531
2939899107,2cyclictest26763-21taskset21:18:481
2939899107,2cyclictest21623-21grep22:15:141
2939899107,2cyclictest20843-21sed00:30:181
2939899107,2cyclictest1823-21sleep23:50:001
2939899107,2cyclictest17048-21grep23:10:431
2939899107,2cyclictest11277-21tail20:45:191
2939899100,9cyclictest51922sleep122:47:161
2939899100,9cyclictest26914-21ntp_states20:10:241
2939399108,1cyclictest32358-21expr23:45:140
2939399108,1cyclictest26967-21basename22:25:160
2939399108,1cyclictest25865-21cut21:15:230
2939399108,1cyclictest23075-21taskset20:04:340
2939399107,2cyclictest4451-21cut19:25:150
2939399107,2cyclictest4446-21gzip20:30:210
2939399107,2cyclictest30657-21grep21:25:140
2939399107,2cyclictest24542-21ntp_states20:05:210
2939399107,2cyclictest2055-21wget19:20:140
2939399107,2cyclictest1945-21cat22:40:190
2939399107,2cyclictest16775-21cat20:55:250
2939399107,2cyclictest14288-21expr19:45:140
2939399107,2cyclictest11863-21apache_volume21:55:140
2939399106,3cyclictest17078-21tail23:10:430
2939399101,5cyclictest4995-21sed22:45:280
2939399100,9cyclictest11512-21apache_volume19:40:240
294009997,1cyclictest8850-21taskset00:01:002
294009997,1cyclictest28177-21taskset22:28:222
294009997,1cyclictest14283-21basename22:00:132
294009996,2cyclictest22451-21grep21:10:142
293989997,1cyclictest6515-21taskset21:40:491
293989997,1cyclictest27349-21taskset23:35:121
293989996,2cyclictest13195-21expr20:50:141
293989992,2cyclictest0-21swapper/100:10:101
293989991,3cyclictest9218-21cron21:50:001
293989990,9cyclictest4841-21ls23:55:131
293939997,1cyclictest30217-21expr23:40:160
293939990,9cyclictest1673-21ntpq20:25:210
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional