You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-10 - 21:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot4s.osadl.org (updated Sat Jan 10, 2026 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3111999644,46cyclictest14022-21ssh10:20:121
3111199575,45cyclictest20812-21ssh11:00:120
3111199554,10cyclictest131rcu_preempt11:30:130
3111199543,4cyclictest131rcu_preempt11:15:130
3111199525,43cyclictest2886-21ssh10:50:140
3111999511,2cyclictest21077-21apt-get08:40:161
3111199514,34cyclictest13064-21ssh11:50:130
3111199512,45cyclictest17835-21ssh09:10:120
3111999495,36cyclictest27813-21apt-get07:40:101
3111999491,5cyclictest0-21swapper/111:55:131
3111999471,3cyclictest10761-21apt-get12:25:161
3111999462,3cyclictest19521-21apt-get08:05:111
3111999452,38cyclictest30804-21apt-get08:50:111
3111999451,2cyclictest9857-21apt-get07:55:111
3112899442,34cyclictest16054-21ssh09:45:132
31128994328,9cyclictest18135-21ssh10:03:552
31111994312,20cyclictest12-21ksoftirqd/012:10:120
3113699425,18cyclictest28366irq/58-eth011:50:133
3113699414,22cyclictest28366irq/58-eth008:00:133
31136994131,6cyclictest0-21swapper/312:30:153
3111999411,35cyclictest0-21swapper/110:05:121
31111994125,7cyclictest12-21ksoftirqd/011:40:130
3112899402,11cyclictest15663-21ssh11:31:552
3111999402,33cyclictest16320-21apt-get12:10:121
3111999401,2cyclictest0-21swapper/108:45:121
3111199402,7cyclictest131rcu_preempt08:40:160
31111994019,9cyclictest12-21ksoftirqd/010:05:130
3111999391,32cyclictest0-21swapper/111:20:131
3111999391,2cyclictest0-21swapper/107:50:121
3111999391,1cyclictest0-21swapper/109:20:091
3111999390,2cyclictest0-21swapper/109:55:121
3111999390,2cyclictest0-21swapper/108:25:121
31136993818,14cyclictest28366irq/58-eth010:25:123
3111999381,2cyclictest0-21swapper/109:50:121
3111999380,2cyclictest0-21swapper/110:55:141
3113699371,20cyclictest28366irq/58-eth010:35:143
3111999371,5cyclictest0-21swapper/107:35:121
3111999371,3cyclictest0-21swapper/109:35:111
3111999371,32cyclictest17418-21apt-get10:40:131
3111999370,5cyclictest14685-21apt-get08:00:141
3111999370,1cyclictest0-21swapper/112:00:121
3111199377,15cyclictest16035-21ssh09:45:120
3113699363,21cyclictest28366irq/58-eth011:00:123
3112899364,24cyclictest1538-21ssh09:55:122
3111999361,2cyclictest3205-21apt-get08:55:121
3111999360,2cyclictest25230-21ssh12:15:121
3111999360,2cyclictest25230-21ssh12:15:121
3112899352,28cyclictest27919-21ssh10:09:562
3111999352,2cyclictest1781-21apt-get08:20:131
3111999352,28cyclictest0-21swapper/111:30:531
3111999351,10cyclictest5992-21uptime12:20:291
3111199357,19cyclictest12-21ksoftirqd/007:25:130
3111199351,29cyclictest17447-21diskmemload10:10:130
3111199351,27cyclictest30308-21ssh09:35:110
3112899346,17cyclictest3676-21kworker/2:011:45:112
3112899342,16cyclictest25467-21ssh11:37:562
3112899342,15cyclictest0-21swapper/210:45:552
3111999342,2cyclictest28767-21apt-get12:35:141
3111999341,4cyclictest32686-21apt-get07:45:131
3111999341,4cyclictest32686-21apt-get07:45:131
3111999341,3cyclictest0-21swapper/108:10:031
3111999341,30cyclictest29776-21kthreadcore10:45:221
3111999341,2cyclictest28357-21apt-get10:10:121
3111999341,2cyclictest15229-21apt-get11:15:121
3111999341,10cyclictest21558-21ntp_192.168.11407:30:251
3111999341,10cyclictest11437-21sed10:55:021
3111999340,2cyclictest22918-21ssh10:25:111
3111199348,20cyclictest12-21ksoftirqd/010:15:110
3111199342,27cyclictest31949-21ssh10:30:140
3112899331,20cyclictest584-21ssh11:25:112
3111999332,28cyclictest0-21swapper/111:25:111
3111999331,9cyclictest1351066ssh09:42:321
3111999331,3cyclictest0-21swapper/112:30:151
3111999331,29cyclictest820-21grep11:05:221
3111999331,10cyclictest25316-21sort09:30:291
3111999331,10cyclictest25316-21sort09:30:281
3111999330,4cyclictest3648-21apt-get07:15:121
3111999330,2cyclictest29339-21apt-get08:15:121
3111999330,2cyclictest0-21swapper/108:35:121
3111999330,10cyclictest5889-21ssh10:15:191
3113699323,17cyclictest28366irq/58-eth011:15:133
3112899322,22cyclictest25035-21ssh09:50:132
3111999322,3cyclictest12896-21apt-get09:05:131
3111999322,2cyclictest8059-21apt-get09:00:141
3111999321,8cyclictest17163-21grep07:25:281
3111999321,8cyclictest13342-21kthreadcore09:25:201
3111999320,2cyclictest13181-21ssh11:50:151
3111999320,2cyclictest0-21swapper/111:45:101
3111999320,2cyclictest0-21swapper/111:40:131
3111999320,2cyclictest0-21swapper/110:30:131
3111999320,1cyclictest0-21swapper/108:30:141
3111199324,19cyclictest3372-21ssh09:20:110
3113699313,20cyclictest28366irq/58-eth009:45:133
3111999312,25cyclictest13261-21latency_hist07:25:021
3111999311,9cyclictest19685-21ntp_192.168.11409:45:241
3111999311,8cyclictest14449-21expr10:00:321
3111999311,6cyclictest27340-21latency_hist11:40:021
3111999311,2cyclictest31233-21apt-get07:10:131
3111999310,10cyclictest22157-21cat09:10:291
3111199312,21cyclictest28787-21ssh12:35:150
3113699307,13cyclictest28366irq/58-eth010:45:143
31136993015,8cyclictest0-21swapper/312:20:103
3112899302,17cyclictest30945-21ssh12:00:102
3111999301,9cyclictest20829-21cpu11:00:151
3111999301,8cyclictest28854-21cut09:15:211
3111999301,6cyclictest31100-21cyclictest07:10:021
3111999301,23cyclictest8742-21fschecks_time10:35:181
3111999300,3cyclictest0-21swapper/111:10:141
3111199309,10cyclictest131rcu_preempt09:15:130
3111199303,5cyclictest131rcu_preempt11:20:120
3111199303,15cyclictest11840-21ssh09:40:320
3111199302,25cyclictest21304-21ssh09:30:130
3111199302,25cyclictest21304-21ssh09:30:130
3113699293,17cyclictest0-21swapper/309:30:563
3113699293,17cyclictest0-21swapper/309:30:563
3112899296,2cyclictest0-21swapper/211:15:132
3112899293,21cyclictest0-21swapper/212:28:562
3112899292,11cyclictest25035-21ssh10:43:562
3112899292,10cyclictest27386-21ssh09:31:562
3112899292,10cyclictest27386-21ssh09:31:552
3111999291,10cyclictest11134-21ntp_states12:05:261
31111992917,9cyclictest8517-21ssh10:35:140
31111992911,12cyclictest31100-21cyclictest10:40:140
3113699284,16cyclictest28366irq/58-eth010:55:123
3112899282,17cyclictest22346irq/40-dwc2_hso08:25:132
3111199283,16cyclictest25255-21ssh12:15:140
3111199283,16cyclictest25255-21ssh12:15:140
3111199282,20cyclictest15971-21ssh12:25:560
3113699265,9cyclictest28366irq/58-eth010:20:123
3112899262,20cyclictest14662-21ssh09:43:552
3112899261,21cyclictest28972-21cut12:35:182
3111199264,20cyclictest19791-21ssh12:30:150
3111199263,12cyclictest1826-21ssh12:20:110
3111199262,19cyclictest18611-21ssh11:35:130
3113699252,17cyclictest0-21swapper/311:30:483
3113699252,17cyclictest0-21swapper/310:09:553
3113699252,15cyclictest28466irq/59-eth012:35:153
3113699252,11cyclictest28366irq/58-eth011:05:123
31136992512,10cyclictest28366irq/58-eth010:00:173
3112899257,15cyclictest23972-21ssh10:25:182
3112899253,13cyclictest29728-21rm11:05:142
3112899251,8cyclictest17447-21diskmemload12:09:562
31128992513,5cyclictest13550-21ssh11:13:282
3112899251,14cyclictest21961-21rm11:55:132
3112899251,11cyclictest14025-21sh10:20:132
3111199255,11cyclictest12348-21ssh09:25:150
31111992510,5cyclictest26317-21ssh10:45:120
3113699246,12cyclictest28366irq/58-eth011:55:133
3113699244,14cyclictest28366irq/58-eth008:35:123
3113699244,14cyclictest0-21swapper/310:15:133
3113699243,16cyclictest0-21swapper/312:25:563
3112899249,12cyclictest13409-21kthreadcore08:30:222
3112899247,12cyclictest27-21ksoftirqd/209:29:562
3112899243,19cyclictest0-21swapper/210:10:122
3112899242,11cyclictest0-21swapper/208:40:152
3112899241,7cyclictest2588866ssh12:15:192
3112899241,7cyclictest2588866ssh12:15:192
3111199242,17cyclictest29727-21sh11:05:140
31111992412,3cyclictest17447-21diskmemload11:28:480
3111199241,16cyclictest14109-21ls10:20:140
3113699234,12cyclictest0-21swapper/311:25:113
3113699232,14cyclictest28366irq/58-eth009:20:103
3112899233,16cyclictest0-21swapper/207:50:142
3112899232,6cyclictest0-21swapper/210:54:572
3112899231,17cyclictest24320-21memory11:00:242
3112899231,14cyclictest4932-21ssh10:15:132
3111199239,4cyclictest131rcu_preempt11:55:170
3111199232,16cyclictest10559-21ssh10:00:170
31111992313,5cyclictest0-21swapper/012:05:130
3111199231,16cyclictest6265-21ssh11:10:130
31136992212,6cyclictest0-21swapper/312:10:133
3112899222,16cyclictest0-21swapper/209:23:572
3112899221,9cyclictest2170166ssh12:30:232
3112899221,10cyclictest8042-21munin-run10:35:022
31128992210,9cyclictest29296-21ssh11:20:492
31128992210,9cyclictest0-21swapper/207:30:182
3112899220,5cyclictest3886-21ls07:15:162
3111199221,19cyclictest4413-21ls11:45:150
3111199221,16cyclictest19702-21grep08:05:150
3111199221,15cyclictest22946-21ssh10:25:120
3113699215,10cyclictest28366irq/58-eth011:35:133
31136992112,5cyclictest0-21swapper/310:10:133
31136992111,6cyclictest28366irq/58-eth010:30:153
3112899219,5cyclictest22678-21ssh12:12:192
3112899218,11cyclictest19505-1kworker/u9:110:55:202
3112899214,9cyclictest1251-21kthreadcore07:10:202
3112899213,15cyclictest8556-21cpuspeed07:20:132
3112899212,16cyclictest376-21fschecks_time07:45:162
3112899212,16cyclictest376-21fschecks_time07:45:162
3112899212,10cyclictest13055-21ssh11:50:112
31128992115,3cyclictest2047-21ssh12:20:162
3112899211,11cyclictest26835-21ssh09:15:142
3112899211,10cyclictest18009-21fschecks_count09:10:162
3111199219,9cyclictest1-21systemd07:14:490
3111199211,6cyclictest91-21pr/ttyS008:25:010
31111992112,6cyclictest0-21swapper/007:45:250
31111992112,6cyclictest0-21swapper/007:45:240
3113699202,6cyclictest0-21swapper/311:44:573
3112899209,8cyclictest15197-21sed07:25:202
3112899204,13cyclictest22346irq/40-dwc2_hso08:10:102
3112899203,8cyclictest30490-21idleruntime-cro08:50:012
3112899203,13cyclictest22346irq/40-dwc2_hso09:00:132
3112899202,9cyclictest18680-21kthreadcore08:35:202
3112899202,15cyclictest13263-21irqcore09:05:182
3112899201,9cyclictest472-21kthreadcore08:50:192
3112899201,9cyclictest30514-21fschecks_time09:35:152
3112899201,9cyclictest2110-21ps08:20:172
3112899201,5cyclictest8035-21apt-get10:35:032
3112899201,16cyclictest29300-21kthreadcore07:40:182
3112899201,16cyclictest27762-21cut11:40:152
3112899201,15cyclictest23080-21fschecks_count07:35:152
3112899201,10cyclictest19721-21grep08:05:152
3112899201,10cyclictest14866-21grep08:00:152
31111992013,5cyclictest9699-21ssh09:59:320
3111199201,12cyclictest0-21swapper/008:15:110
3113699198,6cyclictest0-21swapper/310:44:023
3113699198,6cyclictest0-21swapper/309:38:583
3113699192,11cyclictest0-21swapper/311:45:113
3113699192,10cyclictest0-21swapper/308:15:123
3113699191,13cyclictest0-21swapper/309:40:153
3113699191,12cyclictest0-21swapper/310:50:163
31136991910,6cyclictest28366irq/58-eth009:25:133
3112899191,9cyclictest3539-21ps08:55:162
3112899191,14cyclictest22346irq/40-dwc2_hso08:15:122
3112899191,14cyclictest10832-21kthreadcore07:55:182
3111199192,7cyclictest28108-21kthreadcore09:50:220
3111199191,15cyclictest16836-21kthreadcore08:00:210
3111199191,11cyclictest8229-21latency_hist07:20:020
31111991910,6cyclictest14429-21cron08:00:020
3113699188,8cyclictest0-21swapper/312:18:113
3113699188,8cyclictest0-21swapper/312:18:103
3113699183,10cyclictest0-21swapper/311:10:123
3113699182,10cyclictest0-21swapper/308:55:113
3113699182,10cyclictest0-21swapper/308:20:143
3113699182,10cyclictest0-21swapper/308:05:133
3113699182,10cyclictest0-21swapper/307:50:143
3113699182,10cyclictest0-21swapper/307:35:133
3113699182,10cyclictest0-21swapper/307:30:143
3113699181,11cyclictest0-21swapper/312:05:123
31136991810,6cyclictest0-21swapper/309:14:363
3111199188,7cyclictest12-21ksoftirqd/008:10:110
3111199181,13cyclictest1-21systemd09:05:130
3113699179,6cyclictest28366irq/58-eth009:15:143
3113699179,6cyclictest0-21swapper/311:20:073
3113699175,9cyclictest0-21swapper/309:55:123
3113699172,11cyclictest0-21swapper/309:50:043
3113699172,10cyclictest0-21swapper/308:25:133
3113699172,10cyclictest0-21swapper/307:25:133
3113699172,10cyclictest0-21swapper/307:10:133
31136991711,4cyclictest0-21swapper/312:02:243
3113699171,13cyclictest33-21ksoftirqd/308:30:193
3113699171,13cyclictest0-21swapper/308:40:253
3113699171,11cyclictest0-21swapper/307:55:113
3111199179,6cyclictest25896-21tr07:35:200
3111199179,5cyclictest31643-21irqrtprio12:00:170
3111199177,7cyclictest11101-21cut09:00:220
3111199173,8cyclictest0-21swapper/008:45:120
3111199172,11cyclictest7902-21kthreadcore07:50:200
3111199171,14cyclictest3534-21sort08:55:150
3111199171,13cyclictest11785-21ssh10:55:130
31111991710,4cyclictest0-21swapper/008:25:290
3113699162,10cyclictest0-21swapper/309:05:153
3113699161,12cyclictest0-21swapper/307:45:223
3113699161,12cyclictest0-21swapper/307:45:223
3113699161,10cyclictest0-21swapper/308:10:053
3111199168,5cyclictest14953-21grep08:30:280
3111199167,6cyclictest10537-21kthreadcore07:20:200
3111199162,9cyclictest0-21swapper/007:40:130
3111199162,11cyclictest3036-21grep08:55:010
3111199161,10cyclictest21825-21awk07:30:290
31111991610,4cyclictest0-21swapper/008:35:120
3113699152,9cyclictest0-21swapper/307:20:143
3112899151,11cyclictest31180-21cat07:10:022
3111199151,10cyclictest31143-21wc07:10:020
3113699141,7cyclictest0-21swapper/308:45:283
3113699138,3cyclictest0-21swapper/307:15:113
3113699131,9cyclictest0-21swapper/307:45:023
3113699122,7cyclictest0-21swapper/309:05:013
3113699122,6cyclictest0-21swapper/308:50:043
311369991,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional