You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-28 - 14:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sat Mar 28, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3058999591,46cyclictest30428-21ssh11:40:131
3058199533,36cyclictest20629-21ssh09:30:130
3058999512,39cyclictest1496-21ssh09:55:111
3058999503,29cyclictest17478-21ssh11:15:141
3058999503,29cyclictest17478-21ssh11:15:141
3059799478,9cyclictest131rcu_preempt12:10:112
3058999472,30cyclictest8359-21ssh11:10:131
3058999472,30cyclictest8359-21ssh11:10:131
3060599463,28cyclictest28366irq/58-eth010:35:123
30605994622,13cyclictest28366irq/58-eth009:15:143
3058199460,40cyclictest3098-21apt-get07:15:120
3059799454,21cyclictest24154-21ssh12:30:142
3059799440,34cyclictest0-21swapper/210:40:132
30589994423,15cyclictest21-21ksoftirqd/112:05:131
3058199444,13cyclictest131rcu_preempt09:40:130
3058199442,38cyclictest23680-21apt-get10:25:120
3058199441,33cyclictest22452-21sh09:11:450
3058199440,24cyclictest8952-21ssh10:34:560
3058199432,37cyclictest22358-21apt-get07:35:120
3058199431,33cyclictest16177-21diskmemload11:53:140
3060599423,22cyclictest28366irq/58-eth010:00:123
3058199422,36cyclictest7100-21ssh11:45:120
3058199421,31cyclictest510-21ssh10:46:020
3059799412,21cyclictest15923-21kthreadcore12:25:202
3058199411,36cyclictest0-21swapper/007:55:130
3058199410,37cyclictest23510-21apt-get08:10:160
3058199410,31cyclictest7345-21ssh09:56:360
3058999406,29cyclictest15098-21ssh12:25:141
3058199404,32cyclictest778-21ssh12:35:130
3058199401,34cyclictest0-21swapper/011:20:140
3058199401,32cyclictest0-21swapper/012:05:120
3058199401,29cyclictest4394-21df07:50:150
3058199401,28cyclictest16177-21diskmemload11:56:440
3058199401,25cyclictest0-21swapper/011:34:230
3058199400,24cyclictest16177-21diskmemload11:10:550
3058199400,24cyclictest16177-21diskmemload11:10:550
3058199400,23cyclictest16177-21diskmemload12:01:500
3058199400,23cyclictest0-21swapper/010:24:310
3058199391,25cyclictest29817-21sh12:31:250
3058199391,25cyclictest23913-21ssh09:49:300
3058199391,25cyclictest16177-21diskmemload12:23:320
3058199391,24cyclictest0-21swapper/010:07:220
3058199391,21cyclictest10549-21ssh10:15:450
3058199390,25cyclictest25469-21sh12:11:030
3058199390,23cyclictest19242-21ssh10:04:540
3058199384,31cyclictest11470-21ssh09:25:100
3058199381,31cyclictest30401-21apt-get11:40:130
3058199381,28cyclictest16177-21diskmemload12:15:380
3058199381,24cyclictest21171-21ssh10:58:520
3058199381,23cyclictest4642-21ssh09:38:110
3058199381,21cyclictest11981-21ssh10:53:470
3058199380,24cyclictest9398-21rm11:26:590
3058199380,23cyclictest16482-21sh10:38:080
3060599372,23cyclictest28366irq/58-eth010:55:123
3059799373,26cyclictest7106-21ssh11:45:132
3058199372,30cyclictest25638-21apt-get09:15:130
30581993717,14cyclictest0-21swapper/008:50:140
3058199371,30cyclictest31711-21apt-get11:05:140
3058199371,22cyclictest16813-21grep12:25:200
3058199370,24cyclictest25676-21rm11:19:240
3058199370,24cyclictest25676-21rm11:19:240
3059799365,21cyclictest261rcuc/210:00:142
3058199361,23cyclictest13637-21kworker/u8:311:35:460
3058199361,22cyclictest16177-21diskmemload09:51:160
3058199361,21cyclictest16177-21diskmemload10:11:010
3058199361,20cyclictest31732-21ntp_states08:15:250
3058199361,19cyclictest23278-21latency_hist08:10:010
30581993513,19cyclictest12-21ksoftirqd/008:20:130
3058199351,30cyclictest20045-21apt-get08:40:120
3058199351,30cyclictest20045-21apt-get08:40:120
3058199351,30cyclictest0-21swapper/008:55:140
3058199351,30cyclictest0-21swapper/007:25:130
3058199351,23cyclictest18506-21latency_hist08:05:010
3058199351,23cyclictest18506-21latency_hist08:05:010
3058199351,19cyclictest0-21swapper/010:40:590
3060599342,20cyclictest28366irq/58-eth011:30:123
3059799343,24cyclictest23688-21ssh10:25:132
3059799343,24cyclictest0-21swapper/212:15:192
30597993419,11cyclictest531-21usb-storage08:25:112
3059799341,17cyclictest25649-21ssh09:15:152
3058999342,23cyclictest10651-21ssh10:00:121
3058999340,23cyclictest28832-21ssh10:10:121
3058199341,28cyclictest11593-21apt-get09:05:150
3058199341,23cyclictest5709-21df_inode08:25:120
3058199341,22cyclictest10901-21kthreadcore07:20:220
3058199341,19cyclictest4360-21kthreadcore09:20:210
3060599335,23cyclictest28366irq/58-eth009:20:153
3059799339,9cyclictest5933-21/usr/sbin/munin12:20:192
3059799338,19cyclictest30572-21cyclictest11:50:122
30589993320,5cyclictest5631-21ssh09:39:241
30589993310,8cyclictest131rcu_preempt07:45:251
3058199332,28cyclictest31282-21irqrtprio07:10:180
3058199331,29cyclictest20075-21kthreadcore07:30:230
3058199330,28cyclictest30572-21cyclictest11:00:140
3060599325,18cyclictest28366irq/58-eth010:40:143
3059799326,18cyclictest0-21swapper/212:35:142
3059799323,13cyclictest29855-21ssh09:35:122
3059799321,18cyclictest13430-21ssh10:55:142
3058999322,20cyclictest5978-21ssh12:20:161
3058199321,27cyclictest27181-21/usr/sbin/munin07:40:110
3058199321,26cyclictest0-21swapper/008:35:120
3058199321,18cyclictest0-21swapper/009:00:210
3059799312,23cyclictest16578-21ssh09:10:132
3058999314,17cyclictest18547-21ssh10:40:141
3058999313,15cyclictest19727-21ssh10:05:131
30589993115,10cyclictest16177-21diskmemload12:34:201
3060599304,15cyclictest28366irq/58-eth009:35:113
3060599302,17cyclictest28366irq/58-eth010:25:133
3060599301,18cyclictest28366irq/58-eth012:30:143
3059799302,16cyclictest22346irq/40-dwc2_hso11:10:132
3059799302,16cyclictest22346irq/40-dwc2_hso11:10:132
3058999301,17cyclictest16177-21diskmemload11:05:131
30589993010,11cyclictest0-21swapper/112:00:111
3058199309,18cyclictest29516-21date08:50:010
3058199301,25cyclictest831-21cat07:45:170
3059799294,15cyclictest22346irq/40-dwc2_hso07:30:142
3059799293,15cyclictest319-21ssh10:30:132
3059799292,11cyclictest1396-21ssh09:55:052
3059799291,16cyclictest24738-21ssh09:50:122
3058999292,18cyclictest2370-21ssh09:20:131
3060599283,21cyclictest28366irq/58-eth011:55:133
30589992811,12cyclictest25238-21diskmemload11:55:131
3058199281,24cyclictest13874-21ntp_states08:30:260
3060599271,14cyclictest28366irq/58-eth011:15:143
3060599271,14cyclictest28366irq/58-eth011:15:133
3059799272,18cyclictest11465-21ssh09:25:092
30597992716,6cyclictest22346irq/40-dwc2_hso12:00:102
3058999272,19cyclictest8906-21ssh12:39:181
3060599262,12cyclictest28366irq/58-eth009:45:133
3060599261,17cyclictest28366irq/58-eth012:25:123
3060599261,12cyclictest28366irq/58-eth011:10:123
3060599261,12cyclictest28366irq/58-eth011:10:123
3059799262,9cyclictest26696-21ssh11:55:192
3059799262,18cyclictest0-21swapper/211:40:132
30597992617,6cyclictest23534-21ssh11:35:212
3058999265,10cyclictest666-21ntpd08:30:141
3058999261,18cyclictest29276-21ssh12:15:151
3060599253,13cyclictest0-21swapper/311:45:123
3060599252,13cyclictest28366irq/58-eth011:20:133
3059799253,5cyclictest0-21swapper/208:20:142
3059799252,14cyclictest22346irq/40-dwc2_hso09:05:142
3059799251,17cyclictest9426-21ssh10:35:132
3059799251,13cyclictest19736-21ssh10:05:132
30597992511,10cyclictest11144-21rm12:05:132
3058999259,5cyclictest21-21ksoftirqd/110:51:461
3058999253,14cyclictest15639-21ssh09:45:141
30589992512,5cyclictest21-21ksoftirqd/111:39:421
3058199250,22cyclictest30714-21latency_hist07:10:020
3060599245,13cyclictest28466irq/59-eth009:50:113
3060599241,13cyclictest28366irq/58-eth009:30:133
3059799247,14cyclictest28503-21wc08:15:162
3059799242,17cyclictest0-21swapper/208:50:142
3059799241,10cyclictest6584-21ssh09:40:132
3058999249,5cyclictest21-21ksoftirqd/111:23:011
3058999248,8cyclictest21-21ksoftirqd/111:54:301
3058999248,8cyclictest21-21ksoftirqd/108:25:131
3058999246,8cyclictest21-21ksoftirqd/110:48:501
3058999244,17cyclictest24722-21ssh09:50:111
30589992410,5cyclictest21-21ksoftirqd/111:00:121
3060599233,14cyclictest0-21swapper/310:50:143
3060599232,15cyclictest0-21swapper/312:35:153
30605992315,5cyclictest0-21swapper/309:55:103
30605992312,9cyclictest0-21swapper/310:13:213
3059799239,5cyclictest12307-21find11:30:122
3059799239,11cyclictest13983-21df_inode08:00:162
3059799239,11cyclictest13983-21df_inode08:00:162
3059799233,15cyclictest22346irq/40-dwc2_hso08:35:132
3059799233,15cyclictest22346irq/40-dwc2_hso07:25:142
3059799233,13cyclictest22346irq/40-dwc2_hso08:30:132
3059799232,16cyclictest924-21snmpd08:10:162
3059799232,16cyclictest0-21swapper/207:15:132
3059799231,18cyclictest3259-21ssh11:25:132
3059799231,15cyclictest0-21swapper/209:00:142
3059799231,12cyclictest28752-21cut10:45:202
30597992310,5cyclictest18253-21ssh10:20:282
3059799230,20cyclictest22663-21cpuspeed11:00:142
3058999239,3cyclictest131rcu_preempt09:41:321
3058999238,9cyclictest21-21ksoftirqd/108:35:151
3058999238,9cyclictest0-21swapper/110:15:131
3058999237,7cyclictest21-21ksoftirqd/110:20:441
30589992311,4cyclictest21-21ksoftirqd/108:15:121
3060599225,13cyclictest0-21swapper/311:35:143
3060599223,13cyclictest0-21swapper/309:10:133
3059799223,14cyclictest22346irq/40-dwc2_hso07:35:122
30597992216,4cyclictest4767-21ssh10:50:182
3059799221,14cyclictest31720-21sh11:05:132
3059799221,10cyclictest2465-21df09:20:142
3058999229,9cyclictest12345-21ssh11:30:131
3058999229,5cyclictest21-21ksoftirqd/109:28:231
3058999228,5cyclictest131rcu_preempt11:25:151
3058999227,9cyclictest21-21ksoftirqd/112:14:351
3058999225,8cyclictest21-21ksoftirqd/110:34:131
3058999223,3cyclictest131rcu_preempt09:15:301
3058999222,16cyclictest13404-21apt-get10:55:121
30589992210,3cyclictest131rcu_preempt09:33:111
3059799217,10cyclictest0-21swapper/208:45:202
3059799212,13cyclictest22346irq/40-dwc2_hso09:45:132
3059799212,11cyclictest18235-21kthreadcore11:15:202
3059799212,11cyclictest18235-21kthreadcore11:15:202
3059799212,10cyclictest8399-21irqcore07:20:182
3059799211,16cyclictest5270-21kthreadcore07:50:202
3059799211,11cyclictest8994-21missed_timers10:15:222
3059799211,10cyclictest30810-21kthreadcore10:10:212
30597992110,5cyclictest25785-21ssh09:30:492
3058999217,5cyclictest21-21ksoftirqd/108:10:251
3058999212,4cyclictest21-21ksoftirqd/111:45:201
30589992115,4cyclictest0-21swapper/110:25:121
30589992112,7cyclictest13989-21diskstats08:00:171
30589992112,7cyclictest13989-21diskstats08:00:161
3058999211,11cyclictest5304-21apt-get08:25:031
3058999211,11cyclictest30611-21grep07:10:011
3060599209,8cyclictest0-21swapper/311:43:143
3060599203,12cyclictest28366irq/58-eth008:30:143
3060599202,12cyclictest0-21swapper/312:00:113
30605992013,3cyclictest0-21swapper/310:05:133
3060599201,13cyclictest0-21swapper/309:05:233
30605992010,7cyclictest28366irq/58-eth012:15:143
30605992010,7cyclictest28366irq/58-eth011:25:133
3059799205,6cyclictest31762-21cron07:45:002
3059799202,14cyclictest1085-21kthreadcore07:45:182
3059799202,10cyclictest26591-21/usr/sbin/munin11:20:302
3059799201,14cyclictest18928-21idleruntime08:05:172
3059799201,10cyclictest22615-21kthreadcore08:40:222
3059799201,10cyclictest22615-21kthreadcore08:40:212
3059799201,10cyclictest2175-21head07:10:282
3059799201,10cyclictest10089-21kthreadcore07:55:212
3058999207,7cyclictest21-21ksoftirqd/109:10:031
3058999205,3cyclictest131rcu_preempt07:35:251
3058999205,2cyclictest131rcu_preempt07:10:161
30589992015,3cyclictest0-21swapper/109:05:261
30589992010,3cyclictest21-21ksoftirqd/110:35:491
3060599199,7cyclictest28366irq/58-eth011:05:263
3060599199,7cyclictest0-21swapper/312:05:183
3060599193,11cyclictest0-21swapper/312:20:193
3060599193,11cyclictest0-21swapper/312:13:193
3060599193,11cyclictest0-21swapper/310:45:143
3060599192,12cyclictest0-21swapper/310:15:133
3059799191,14cyclictest0-21swapper/208:55:142
3058999199,2cyclictest21-21ksoftirqd/107:50:131
3058999198,3cyclictest131rcu_preempt09:00:271
3058999198,3cyclictest131rcu_preempt08:55:321
3058999196,4cyclictest131rcu_preempt08:05:291
3058999192,5cyclictest16452-21grep07:25:281
3058999192,3cyclictest0-21swapper/107:30:281
3058999191,4cyclictest131rcu_preempt07:40:131
30589991913,4cyclictest0-21swapper/108:45:121
3060599189,6cyclictest28366irq/58-eth011:50:133
3060599189,6cyclictest28366irq/58-eth011:00:143
3060599182,11cyclictest0-21swapper/308:55:153
3060599182,10cyclictest0-21swapper/308:20:053
3060599182,10cyclictest0-21swapper/308:10:163
3060599182,10cyclictest0-21swapper/308:05:143
3060599182,10cyclictest0-21swapper/307:20:153
30605991812,4cyclictest0-21swapper/309:27:123
3060599181,14cyclictest0-21swapper/309:40:213
3060599181,12cyclictest0-21swapper/307:10:163
30605991810,5cyclictest28366irq/58-eth010:30:133
3059799183,6cyclictest30617-21latency_hist07:10:012
3058999186,3cyclictest131rcu_preempt07:55:161
3058999185,6cyclictest21-21ksoftirqd/108:50:151
3058999181,5cyclictest21-21ksoftirqd/108:40:131
3058999181,5cyclictest21-21ksoftirqd/108:40:131
30589991813,3cyclictest0-21swapper/107:15:141
3060599179,6cyclictest0-21swapper/310:21:383
3060599172,10cyclictest0-21swapper/309:00:143
3060599172,10cyclictest0-21swapper/307:35:133
3058999179,6cyclictest7957-21apt-get07:20:131
3060599162,10cyclictest0-21swapper/307:40:143
3060599161,12cyclictest0-21swapper/308:45:223
3060599161,12cyclictest0-21swapper/308:25:133
3060599161,12cyclictest0-21swapper/307:50:183
3060599161,10cyclictest0-21swapper/308:40:193
3060599161,10cyclictest0-21swapper/308:40:193
30605991610,4cyclictest28366irq/58-eth007:55:143
3060599141,7cyclictest0-21swapper/307:15:223
3060599131,9cyclictest0-21swapper/308:35:193
3060599131,8cyclictest28366irq/58-eth007:30:273
3060599131,10cyclictest0-21swapper/308:00:153
3060599131,10cyclictest0-21swapper/308:00:153
3060599131,10cyclictest0-21swapper/307:45:133
3060599123,7cyclictest0-21swapper/307:25:293
3060599121,8cyclictest0-21swapper/308:15:133
3060599121,5cyclictest0-21swapper/308:50:143
3060599101,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional