You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-21 - 18:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Aug 21, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2310699756,52cyclictest16119-21ssh09:50:121
2309299743,55cyclictest3811-21ssh10:55:140
2309299743,55cyclictest3811-21ssh10:55:130
2311599681,34cyclictest14669-21ssh11:55:172
2309299607,6cyclictest131rcu_preempt11:20:130
2311599550,33cyclictest0-21swapper/210:25:122
2310699531,41cyclictest9037-21ssh09:10:121
2310699522,37cyclictest25495-21ssh11:25:141
2309299511,37cyclictest19715-21ssh10:10:130
2312799495,27cyclictest35966irq/58-eth010:55:133
2312799495,27cyclictest35966irq/58-eth010:55:123
23127994834,9cyclictest35966irq/58-eth011:35:123
2310699473,35cyclictest25176-21ssh09:55:111
2310699473,2cyclictest0-21swapper/111:50:141
2310699471,3cyclictest3665-21ssh09:25:151
23092994733,8cyclictest0-21swapper/011:40:130
23127994613,22cyclictest35966irq/58-eth010:30:123
2311599453,31cyclictest11116-21ssh11:35:112
2310699451,4cyclictest0-21swapper/110:05:141
2309299451,33cyclictest9116-21ssh12:10:120
2310699441,3cyclictest0-21swapper/112:25:101
2310699441,3cyclictest0-21swapper/108:50:131
2310699441,3cyclictest0-21swapper/108:50:121
2310699441,38cyclictest27079-21ssh09:20:151
2310699432,3cyclictest0-21swapper/108:00:131
2310699431,4cyclictest0-21swapper/109:45:121
2310699431,3cyclictest1643-21apt-get07:55:121
2309299433,24cyclictest21849-21ssh11:05:120
2309299423,8cyclictest131rcu_preempt07:50:110
2310699411,2cyclictest5426-21apt-get10:20:111
2312799405,22cyclictest35966irq/58-eth012:25:123
2312799405,20cyclictest35966irq/58-eth009:50:123
2311599403,29cyclictest12753-21ssh12:30:122
2310699401,4cyclictest0-21swapper/107:40:131
2310699401,3cyclictest0-21swapper/109:05:131
2310699401,2cyclictest0-21swapper/112:30:131
2310699400,6cyclictest19701-21apt-get10:10:111
2309299405,25cyclictest29023-21ssh11:45:130
2309299402,29cyclictest27192-21ssh12:20:120
2312799396,27cyclictest35966irq/58-eth009:45:133
2312799392,22cyclictest35966irq/58-eth011:00:113
2310699391,3cyclictest0-21swapper/108:55:131
2310699390,3cyclictest28779-21ssh10:15:131
2309299393,25cyclictest12-21ksoftirqd/008:40:120
2312799387,20cyclictest35966irq/58-eth009:15:133
2311599384,27cyclictest5553-21ssh11:50:132
2311599383,26cyclictest3656-21ssh09:25:142
2310699383,2cyclictest20088-21apt-get11:40:121
2310699382,3cyclictest29017-21apt-get11:45:131
2310699381,3cyclictest0-21swapper/111:35:131
2309299381,23cyclictest12-21ksoftirqd/007:45:140
2309299381,23cyclictest12-21ksoftirqd/007:45:140
2310699371,4cyclictest0-21swapper/111:05:131
2310699371,10cyclictest11238-21apt-get08:05:111
2310699370,4cyclictest392-21apt-get07:20:111
23092993716,13cyclictest12-21ksoftirqd/012:00:120
2310699361,4cyclictest30601-21apt-get09:40:161
2310699360,4cyclictest7415-21apt-get11:15:131
2309299361,5cyclictest131rcu_preempt10:20:120
23092993615,7cyclictest131rcu_preempt07:15:120
2309299360,25cyclictest27260-21ssh10:50:110
2312799358,16cyclictest35966irq/58-eth010:35:123
2312799353,20cyclictest35966irq/58-eth012:05:153
2312799352,19cyclictest35966irq/58-eth012:00:133
2311599353,26cyclictest0-21swapper/212:18:562
2311599353,26cyclictest0-21swapper/212:18:562
2310699358,23cyclictest15096-21tr09:30:211
2310699351,9cyclictest17923-21cut08:45:181
2310699351,3cyclictest0-21swapper/107:30:111
2309299355,18cyclictest18213-21ssh10:45:120
23127993420,6cyclictest0-21swapper/312:35:113
2310699345,25cyclictest5519-21kthreadcore10:55:211
2310699345,25cyclictest5519-21kthreadcore10:55:201
2310699341,4cyclictest21608-21apt-get09:35:131
2310699341,2cyclictest24476-21apt-get07:45:141
2310699341,2cyclictest24476-21apt-get07:45:141
2310699341,24cyclictest21739-21sh12:35:101
2310699341,10cyclictest494466ssh10:35:311
23115993329,2cyclictest4623-21ssh11:49:222
2310699336,23cyclictest12797-21ntp_offset12:10:251
2310699335,24cyclictest31306-21unixbench_multi10:50:281
2310699335,24cyclictest19572-21kthreadcore12:15:201
2310699335,24cyclictest19572-21kthreadcore12:15:191
2310699331,9cyclictest28489-21tr08:20:201
2310699331,8cyclictest20177-21ssh11:20:261
2310699331,7cyclictest15862-21kthreadcore10:25:201
2310699331,7cyclictest14850-21fschecks_time11:55:181
2310699331,5cyclictest23687-21df_inode10:30:141
2310699330,4cyclictest23632-21apt-get12:00:131
2312799323,21cyclictest0-21swapper/310:47:573
2311599324,19cyclictest23074-21cyclictest09:05:132
2310699322,21cyclictest32597-21ssh12:05:141
2310699321,9cyclictest7810-21df_inode08:35:161
2310699321,7cyclictest31661-21kthreadcore08:25:181
2310699321,7cyclictest27365-21fschecks_count12:20:161
2310699321,7cyclictest22101-21switchtime10:45:281
2310699321,6cyclictest31647-21wc07:15:261
2310699321,6cyclictest13756-21kthreadcore11:00:181
2310699321,4cyclictest12923-21ntp_states10:40:251
2310699321,4cyclictest0-21swapper/109:00:121
2310699321,3cyclictest1911-21ssh10:00:151
2310699321,23cyclictest18012-21ssh09:15:141
2310699321,10cyclictest4820-21tr11:30:201
23092993219,6cyclictest131rcu_preempt08:05:120
2311599312,24cyclictest19706-21ssh10:10:132
2310699312,6cyclictest17721-21cut07:35:211
2310699311,8cyclictest31427-21cut07:50:201
2310699311,7cyclictest31489-21sed11:10:191
2310699311,7cyclictest2902-21/usr/sbin/munin08:30:291
2310699311,7cyclictest22390-21cut08:15:201
2310699311,7cyclictest22390-21cut08:15:191
2310699311,6cyclictest7889-21tr07:25:201
2310699311,5cyclictest16016-21grep08:40:261
2312799301,18cyclictest35966irq/58-eth010:25:123
2310699301,8cyclictest23250-21/usr/sbin/munin07:10:111
2310699301,7cyclictest19854-21unixbench_singl08:10:291
2312799292,16cyclictest35966irq/58-eth008:45:123
2311599292,18cyclictest0-21swapper/209:45:142
2311599285,17cyclictest3734-21ssh12:25:122
2311599284,19cyclictest0-21swapper/209:55:122
2311599284,16cyclictest27346irq/40-dwc2_hso10:40:132
2311599283,22cyclictest7439-21sh11:15:132
2311599281,18cyclictest24658-21cut07:45:162
2311599281,18cyclictest24658-21cut07:45:162
2309299282,20cyclictest2396-21ssh09:24:000
2312799273,16cyclictest0-21swapper/309:24:003
2311599279,15cyclictest27-21ksoftirqd/210:55:132
2311599279,15cyclictest27-21ksoftirqd/210:55:132
2311599273,19cyclictest0-21swapper/210:50:572
2311599272,19cyclictest12730-21ssh12:10:252
2311599272,10cyclictest30628-21ssh09:40:162
2311599272,10cyclictest0-21swapper/207:10:122
23115992715,10cyclictest1781-21sh10:00:132
23115992713,11cyclictest27-21ksoftirqd/207:35:142
2311599271,15cyclictest2065-21ssh11:30:132
2309299275,15cyclictest151rcuc/009:40:150
2312799262,16cyclictest35966irq/58-eth011:05:123
2312799261,14cyclictest35966irq/58-eth009:40:153
2310699261,7cyclictest23109-21latency_hist07:10:011
2311599253,16cyclictest27346irq/40-dwc2_hso09:00:122
2311599253,14cyclictest27346irq/40-dwc2_hso07:55:152
2309299251,4cyclictest131rcu_preempt08:35:140
2309299251,16cyclictest21867-21ssh12:35:140
2312799242,13cyclictest35966irq/58-eth011:25:143
2311599244,16cyclictest27346irq/40-dwc2_hso10:45:112
2311599243,19cyclictest0-21swapper/209:23:592
2311599243,14cyclictest27346irq/40-dwc2_hso08:45:132
2311599242,18cyclictest27807-21ssh12:36:562
2311599242,18cyclictest21377-21ssh11:20:342
2311599242,15cyclictest27346irq/40-dwc2_hso08:40:122
2311599242,13cyclictest5446-21ssh10:20:142
23115992418,3cyclictest23637-21ssh12:00:132
23115992412,9cyclictest6534-21ssh12:07:222
2311599241,19cyclictest27346irq/40-dwc2_hso10:05:142
2309299243,17cyclictest18033-21ssh09:15:150
2312799233,13cyclictest0-21swapper/312:18:563
2312799233,13cyclictest0-21swapper/312:18:553
23127992312,9cyclictest0-21swapper/311:42:113
2312799231,10cyclictest35966irq/58-eth007:10:113
2311599234,9cyclictest23179-21latency_hist07:10:022
2311599232,13cyclictest20097-21sh11:40:122
2311599232,12cyclictest25127-21kthreadcore11:05:222
2311599231,17cyclictest5205-21find07:25:112
2311599231,15cyclictest27346irq/40-dwc2_hso07:30:122
2311599231,13cyclictest21624-21ssh09:35:122
23092992314,5cyclictest131rcu_preempt10:05:130
2312799228,9cyclictest35966irq/58-eth010:15:143
23127992216,4cyclictest321rcuc/310:50:113
23127992216,4cyclictest321rcuc/309:14:013
23127992216,4cyclictest0-21swapper/311:45:163
23127992214,4cyclictest0-21swapper/309:30:133
23127992212,8cyclictest0-21swapper/311:15:123
23127992212,7cyclictest0-21swapper/309:55:113
2311599228,7cyclictest6677-21fw_packets08:00:162
2311599223,14cyclictest0-21swapper/210:35:112
2311599222,10cyclictest4260-21kthreadcore08:30:202
2311599221,9cyclictest22216-21ssh09:51:572
2311599221,17cyclictest16345-21sort11:00:222
2309299229,9cyclictest1375-21cron10:00:000
2309299222,3cyclictest131rcu_preempt11:15:230
2309299222,14cyclictest17540-21idleruntime08:45:160
2309299222,14cyclictest12301-21ssh12:29:560
23092992213,5cyclictest1798-21sh10:00:140
23092992212,7cyclictest12-21ksoftirqd/012:15:120
23092992212,7cyclictest12-21ksoftirqd/012:15:110
2312799213,12cyclictest0-21swapper/310:10:133
23127992115,4cyclictest0-21swapper/311:30:323
23127992115,4cyclictest0-21swapper/309:05:233
23127992115,4cyclictest0-21swapper/309:00:163
23127992114,5cyclictest0-21swapper/307:25:233
23127992112,7cyclictest0-21swapper/312:21:503
2311599218,4cyclictest28837-21kthreadcore12:20:202
2311599216,3cyclictest877-21lldpd07:43:452
2311599216,12cyclictest28722-21kthreadcore07:15:192
2311599216,12cyclictest0-21swapper/208:55:012
2311599216,12cyclictest0-21swapper/208:55:012
2311599212,12cyclictest0-21swapper/208:05:132
2311599212,10cyclictest26202-21kthreadcore10:30:222
2311599211,17cyclictest15130-21cut09:30:202
2311599211,15cyclictest31107-21cut11:10:162
2311599211,11cyclictest32542-21cat07:50:222
2311599211,11cyclictest20932-21sort08:15:162
2311599211,11cyclictest20932-21sort08:15:152
2311599211,11cyclictest11000-21kthreadcore09:10:192
2311599211,10cyclictest12358-21latency_hist08:40:012
2309299212,14cyclictest23028-21ssh10:29:550
2309299212,13cyclictest9610-21ssh09:26:550
2309299211,16cyclictest19658-21apt-get07:40:110
2309299211,16cyclictest16193-21ssh09:30:240
2309299211,15cyclictest32484-21ssh10:15:250
2309299211,15cyclictest17678-21ls09:15:020
2309299210,6cyclictest18060-21ssh11:38:030
23127992014,4cyclictest0-21swapper/311:50:133
23127992014,4cyclictest0-21swapper/311:10:273
23127992014,4cyclictest0-21swapper/310:40:303
23127992014,4cyclictest0-21swapper/310:20:273
23127992014,4cyclictest0-21swapper/310:01:443
23127992014,4cyclictest0-21swapper/307:40:213
23127992014,4cyclictest0-21swapper/307:35:243
23127992013,5cyclictest321rcuc/308:50:133
23127992013,5cyclictest321rcuc/308:50:133
23127992013,4cyclictest0-21swapper/310:07:393
23127992012,5cyclictest0-21swapper/311:20:133
2312799201,14cyclictest0-21swapper/309:25:133
2312799201,12cyclictest35966irq/58-eth008:25:113
2311599204,9cyclictest18977-21kthreadcore09:15:202
2311599204,13cyclictest27346irq/40-dwc2_hso08:20:122
2311599202,9cyclictest30650-21fschecks_count08:25:142
2311599201,9cyclictest28928-21fschecks_count10:15:152
2311599201,9cyclictest27117-21cut08:55:172
23115992014,4cyclictest25740-21ssh11:25:162
23115992011,7cyclictest20541-21vmstat08:10:312
2311599200,15cyclictest383-21find07:20:102
2309299202,15cyclictest7124-21grep11:15:020
2309299202,12cyclictest23046-21ssh09:52:570
23092992011,6cyclictest0-21swapper/008:25:020
2309299201,15cyclictest32757-21cut12:05:170
2309299201,10cyclictest3263-21kthreadcore07:55:210
2309299200,8cyclictest25648-21fschecks_count11:25:150
2312799192,11cyclictest0-21swapper/308:55:133
23127991914,3cyclictest0-21swapper/312:30:233
23127991914,3cyclictest0-21swapper/308:00:163
23127991913,4cyclictest0-21swapper/311:55:353
23127991913,4cyclictest0-21swapper/309:35:233
23127991913,4cyclictest0-21swapper/308:35:193
23127991913,4cyclictest0-21swapper/308:30:113
23127991913,4cyclictest0-21swapper/308:05:223
2312799191,13cyclictest0-21swapper/312:10:123
2309299197,9cyclictest614-21dbus-daemon11:34:200
2309299191,3cyclictest0-21swapper/009:36:170
2309299191,2cyclictest131rcu_preempt10:30:150
23092991912,5cyclictest12-21ksoftirqd/010:40:020
2309299191,12cyclictest12659-21ssh10:40:220
2309299190,9cyclictest0-21swapper/008:15:190
2309299190,9cyclictest0-21swapper/008:15:190
23127991812,4cyclictest0-21swapper/308:20:183
23127991812,4cyclictest0-21swapper/308:15:193
23127991812,4cyclictest0-21swapper/308:15:193
23127991812,4cyclictest0-21swapper/308:10:153
23127991812,4cyclictest0-21swapper/307:55:253
23127991812,4cyclictest0-21swapper/307:45:143
23127991812,4cyclictest0-21swapper/307:45:133
23127991812,4cyclictest0-21swapper/307:20:193
2309299187,3cyclictest12-21ksoftirqd/008:25:110
2309299185,10cyclictest5149-21kthreadcore08:30:200
2309299182,3cyclictest131rcu_preempt11:55:280
2309299182,11cyclictest0-21swapper/007:20:120
2309299181,9cyclictest2302-21tr09:00:220
2309299181,7cyclictest9801-21munin-run07:30:020
2309299181,7cyclictest15316-21kthreadcore11:00:200
2309299181,4cyclictest131rcu_preempt11:50:210
2309299181,14cyclictest7769-21sed09:45:170
2309299181,14cyclictest5375-21kthreadcore09:05:180
23127991712,3cyclictest321rcuc/308:40:133
23127991712,3cyclictest0-21swapper/307:15:173
23127991711,4cyclictest0-21swapper/307:30:183
2309299179,5cyclictest13667-21kthreadcore12:30:180
2309299173,11cyclictest19405-21cron07:40:010
2309299171,8cyclictest0-21swapper/008:55:300
2309299171,4cyclictest0-21swapper/008:53:280
2309299171,4cyclictest0-21swapper/008:53:270
2309299171,13cyclictest6671-21tune2fs08:00:150
23127991611,3cyclictest0-21swapper/307:50:143
2309299169,4cyclictest16373-21irqcore08:10:170
2309299161,12cyclictest23769-21sed07:10:170
2309299161,12cyclictest10026-21apt-get07:30:130
2309299141,10cyclictest23118-21latency_hist07:10:020
2312799111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional