You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-05 - 23:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Dec 05, 2025 12:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
85799534,45cyclictest13404-21ssh11:20:130
85799514,42cyclictest3656-21ssh11:35:100
857994630,10cyclictest0-21swapper/009:40:130
87599451,36cyclictest26666-21ssh09:20:112
86799452,28cyclictest24865-21ssh12:30:141
85799452,36cyclictest24082-21ssh11:05:130
88399444,24cyclictest29566irq/58-eth010:35:133
88399442,26cyclictest29566irq/58-eth010:05:143
875994320,15cyclictest27-21ksoftirqd/210:25:132
86799432,33cyclictest5819-21rm10:10:131
87599422,21cyclictest17197-21ssh12:05:152
87599421,25cyclictest7620-21ssh09:50:122
86799421,29cyclictest8591-21ssh12:20:131
88399413,27cyclictest29566irq/58-eth011:15:133
883994116,14cyclictest29566irq/58-eth011:35:113
857994114,20cyclictest12-21ksoftirqd/012:10:140
88399403,19cyclictest29566irq/58-eth009:25:133
875994025,9cyclictest4357-21ssh11:12:312
857994023,13cyclictest0-21swapper/009:45:120
87599393,27cyclictest13573-21ssh10:15:132
87599391,24cyclictest27-21ksoftirqd/210:45:142
875993817,18cyclictest27-21ksoftirqd/212:10:152
88399371,21cyclictest29566irq/58-eth011:55:133
867993716,10cyclictest844-21cyclictest07:55:121
86799371,4cyclictest131rcu_preempt11:45:151
85799373,23cyclictest27083-21ssh10:45:140
87599362,27cyclictest29399-21cat09:40:312
85799363,20cyclictest842-21munin-node10:15:130
883993524,6cyclictest29566irq/58-eth011:30:153
86799358,17cyclictest21-21ksoftirqd/110:00:161
85799352,27cyclictest468-21ssh12:35:110
85799351,29cyclictest19358-21ssh10:40:110
88399342,17cyclictest29566irq/58-eth007:25:133
88399341,19cyclictest29566irq/58-eth012:25:153
86799333,25cyclictest9793-21ssh12:00:111
88399325,12cyclictest29566irq/58-eth011:05:133
883993213,13cyclictest0-21swapper/310:45:143
87599323,18cyclictest32506-21ssh09:45:122
87599321,22cyclictest0-21swapper/208:10:122
86799323,19cyclictest20753-21ssh11:25:141
86799323,18cyclictest30724-21ssh10:05:151
883993115,11cyclictest29566irq/58-eth011:25:143
88399311,14cyclictest29566irq/58-eth012:10:153
87599317,12cyclictest844-21cyclictest10:10:112
87599312,22cyclictest14618-21ssh10:56:302
88399303,14cyclictest29566irq/58-eth009:35:113
88399302,17cyclictest0-21swapper/309:34:363
87599302,22cyclictest4555-21cat12:15:302
87599302,22cyclictest24506-21ssh12:30:012
86799302,22cyclictest30112-21ssh09:41:311
88399292,15cyclictest29566irq/58-eth011:40:123
87599299,11cyclictest27-21ksoftirqd/210:40:112
87599294,18cyclictest26346irq/40-dwc2_hso07:40:132
87599282,20cyclictest0-21swapper/211:05:142
87599281,12cyclictest23055-21ssh10:00:162
86799283,19cyclictest24101-21ssh11:05:141
88399273,15cyclictest0-21swapper/311:12:313
883992711,11cyclictest29566irq/58-eth010:00:173
87599273,20cyclictest0-21swapper/209:39:312
87599272,20cyclictest0-21swapper/209:32:302
875992711,12cyclictest629-21Async14:00:482
86799273,18cyclictest201rcuc/111:30:151
857992720,3cyclictest28879-21ssh10:25:120
88399263,15cyclictest0-21swapper/312:15:303
88399262,14cyclictest29566irq/58-eth007:40:103
87599269,9cyclictest18917-21cron09:15:012
87599263,13cyclictest26346irq/40-dwc2_hso11:40:122
87599261,15cyclictest0-21swapper/208:55:112
875992611,11cyclictest0-21swapper/211:25:172
87599260,8cyclictest17056-21fschecks_count11:00:162
867992617,3cyclictest0-21swapper/109:10:111
85799262,19cyclictest11844-21ssh10:35:140
85799262,18cyclictest5964-21ssh11:15:150
88399252,11cyclictest0-21swapper/312:04:303
88399251,14cyclictest0-21swapper/309:40:123
87599252,16cyclictest26346irq/40-dwc2_hso08:25:132
87599252,16cyclictest0-21swapper/211:35:122
87599251,14cyclictest30728-21ssh10:05:152
86799252,18cyclictest15568-21ssh09:33:301
85799252,3cyclictest131rcu_preempt09:35:100
85799251,18cyclictest24396-21ssh11:25:260
85799251,11cyclictest13055-21tr07:40:180
88399242,17cyclictest0-21swapper/310:19:423
87599244,14cyclictest11161-21diskmemload12:35:012
87599241,18cyclictest467-21diskmemload12:35:112
86799244,16cyclictest4211-21ssh10:30:121
86799242,19cyclictest6477-21ssh12:17:291
86799242,16cyclictest1928-21ssh09:25:151
85799246,3cyclictest131rcu_preempt10:20:130
88399239,9cyclictest29566irq/58-eth009:50:123
87599238,5cyclictest28556-21sh10:25:012
87599232,16cyclictest26346irq/40-dwc2_hso10:35:142
875992311,9cyclictest0-21swapper/208:40:232
87599231,16cyclictest0-21swapper/207:30:052
86799231,16cyclictest15401-21ssh09:55:151
86799231,16cyclictest15401-21ssh09:55:141
85799237,4cyclictest131rcu_preempt09:55:150
85799237,4cyclictest131rcu_preempt09:55:150
85799232,15cyclictest12-21ksoftirqd/008:55:110
87599225,10cyclictest23170-21sh11:45:262
87599222,15cyclictest0-21swapper/212:03:302
87599221,8cyclictest0-21swapper/209:25:152
875992211,9cyclictest11238-21ssh11:16:262
86799224,12cyclictest9603-21ssh10:55:131
86799221,17cyclictest19399-21cut09:15:151
85799222,15cyclictest7625-21ssh09:50:120
88399219,9cyclictest0-21swapper/312:30:223
88399213,14cyclictest0-21swapper/309:45:183
883992114,4cyclictest321rcuc/309:25:003
883992113,5cyclictest0-21swapper/309:19:393
883992112,7cyclictest0-21swapper/309:13:053
87599217,5cyclictest16671-21kthreadcore09:55:212
87599217,5cyclictest16671-21kthreadcore09:55:212
87599215,13cyclictest1-21systemd09:09:292
87599212,10cyclictest19379-21fschecks_count09:15:152
87599211,7cyclictest91-21pr/ttyS012:00:012
87599211,17cyclictest11432-21apt-config10:35:012
875992110,3cyclictest0-21swapper/207:25:172
86799211,9cyclictest20311-21idleruntime-cro11:25:021
867992111,7cyclictest4164-21unixbench_singl09:45:281
85799219,10cyclictest32711-21cron08:35:020
85799212,17cyclictest8600-21ssh12:20:140
85799212,13cyclictest842-21munin-node11:30:140
85799211,16cyclictest7765-21kthreadcore10:10:210
85799211,15cyclictest30339-21ssh09:20:240
85799211,10cyclictest1474-21cut08:40:190
88399209,6cyclictest29566irq/58-eth009:55:143
88399209,6cyclictest29566irq/58-eth009:55:143
88399203,12cyclictest0-21swapper/310:14:373
883992014,4cyclictest0-21swapper/312:23:423
883992014,4cyclictest0-21swapper/311:49:393
883992014,4cyclictest0-21swapper/311:01:083
883992013,5cyclictest0-21swapper/312:35:113
883992013,5cyclictest0-21swapper/310:57:063
883992012,5cyclictest0-21swapper/312:05:173
87599207,5cyclictest3289-21cat10:50:202
87599201,9cyclictest27676-21latency_hist08:25:012
87599201,8cyclictest0-21swapper/211:20:132
875992014,4cyclictest9074-21ssh12:20:182
86799209,8cyclictest135-21systemd-journal10:25:011
86799209,3cyclictest131rcu_preempt07:35:011
86799208,9cyclictest491-21cron08:40:011
86799208,9cyclictest12632-21tr10:35:201
86799208,2cyclictest131rcu_preempt11:15:211
86799207,4cyclictest131rcu_preempt09:20:271
867992012,4cyclictest19556-21kthreadcore12:05:221
86799201,16cyclictest5809-21latency_hist11:15:011
86799200,17cyclictest27943-21cut08:25:151
85799201,16cyclictest9803-21ssh12:00:120
85799201,13cyclictest17196-21ssh12:05:150
85799201,10cyclictest22818-21ntp_192.168.11409:15:230
857992010,8cyclictest0-21swapper/012:34:380
857992010,7cyclictest18027-21cut11:00:200
883991913,4cyclictest0-21swapper/311:54:573
883991913,4cyclictest0-21swapper/311:20:243
883991913,4cyclictest0-21swapper/310:41:533
883991913,4cyclictest0-21swapper/310:31:273
883991913,4cyclictest0-21swapper/310:21:223
88399191,12cyclictest29566irq/58-eth010:25:153
88399191,12cyclictest0-21swapper/307:40:013
87599195,7cyclictest27-21ksoftirqd/211:50:152
87599191,9cyclictest387-21latency_hist08:35:022
87599191,8cyclictest20097-21kthreadcore07:55:192
87599191,7cyclictest93266snmpd08:39:262
875991912,5cyclictest1-21systemd08:49:272
87599191,14cyclictest6280-21latency_hist07:25:022
87599191,10cyclictest16637-21cron07:50:002
86799196,4cyclictest131rcu_preempt07:30:011
86799196,10cyclictest11405-21cat09:50:261
86799192,12cyclictest20533-21ssh11:00:261
86799192,10cyclictest2213-21ssh10:50:141
867991911,6cyclictest11541-21sshd07:39:301
86799190,16cyclictest27023-21ls08:15:001
85799192,13cyclictest2025-21df_inode09:25:150
85799191,15cyclictest19595-21sh11:45:160
857991911,5cyclictest15975-21sh09:10:270
883991813,3cyclictest0-21swapper/310:50:203
87599181,9cyclictest22096-21latency_hist08:05:012
87599181,8cyclictest1797-21kthreadcore07:10:172
87599181,8cyclictest16887-21grep07:55:012
87599180,10cyclictest22158-21sh08:10:022
86799189,6cyclictest31681-21cat10:45:321
86799189,6cyclictest14118-21ps10:15:171
86799188,7cyclictest22788-21cut10:40:201
86799182,9cyclictest2356-21ssh11:55:131
86799181,12cyclictest19456-21ssh12:25:221
867991810,6cyclictest5852-21cron07:17:021
867991810,6cyclictest10658-21cron09:00:021
86799180,11cyclictest449-21ssh12:35:101
85799189,5cyclictest2249-21kthreadcore12:15:220
85799188,7cyclictest27493-21cron08:25:010
85799188,7cyclictest15913-21cat11:40:290
85799187,3cyclictest12-21ksoftirqd/007:55:160
85799181,9cyclictest30828-21ssh10:05:160
85799181,7cyclictest0-21swapper/012:25:210
85799181,14cyclictest2346-21apt-get11:55:120
85799181,14cyclictest23032-21apt-get10:00:160
85799181,14cyclictest22545-21uniq08:10:160
85799181,13cyclictest2330-21sh10:50:160
857991810,5cyclictest31463-21timerandwakeup11:50:290
88399172,9cyclictest0-21swapper/308:30:043
883991712,3cyclictest0-21swapper/308:40:233
883991712,3cyclictest0-21swapper/308:25:033
883991712,3cyclictest0-21swapper/307:30:303
883991712,3cyclictest0-21swapper/307:10:163
883991711,4cyclictest0-21swapper/308:55:263
883991711,4cyclictest0-21swapper/307:20:023
87599171,9cyclictest5887-21cat07:20:012
87599171,13cyclictest27249-21munin-run08:20:012
87599171,12cyclictest972-21cat07:10:012
875991710,5cyclictest1-21systemd09:04:282
86799179,5cyclictest16633-21sshd07:49:301
86799178,6cyclictest3472-21kthreadcore08:40:221
86799172,12cyclictest0-21swapper/111:40:131
86799171,13cyclictest7976-21cat11:35:281
867991711,3cyclictest32399-21apt-get08:30:041
86799171,12cyclictest5719-21latency_hist08:50:021
86799171,12cyclictest28972-21df10:25:151
86799171,11cyclictest24648-21/usr/sbin/munin12:10:221
85799179,5cyclictest6184-21latency_hist07:25:020
85799179,5cyclictest4358-21sshd11:12:300
85799177,8cyclictest9710-21df_inode10:55:160
85799175,9cyclictest1532-21ps07:10:160
85799172,12cyclictest9118-21ssh09:30:150
85799172,12cyclictest891-21grep07:10:010
85799171,13cyclictest27497-21apt-get08:25:040
85799171,12cyclictest4309-21diskstats10:30:160
883991611,3cyclictest0-21swapper/308:10:223
883991610,4cyclictest321rcuc/307:55:113
87599160,9cyclictest856-21sshd07:35:292
86799169,5cyclictest5621-21sshd07:14:061
86799168,6cyclictest29490-21cut11:50:221
86799168,5cyclictest5856-21cron08:55:011
86799165,8cyclictest21578-21cat09:35:281
86799161,11cyclictest15507-21timerandwakeup07:40:261
86799160,13cyclictest6136-21munin-plugin-st07:25:001
85799169,5cyclictest12-21ksoftirqd/008:40:000
85799168,6cyclictest5612-21sshd08:49:300
85799161,4cyclictest0-21swapper/008:03:030
85799161,11cyclictest11158-21cron09:10:010
88399157,5cyclictest0-21swapper/308:10:023
88399151,11cyclictest0-21swapper/309:10:023
883991510,3cyclictest0-21swapper/308:36:293
883991510,3cyclictest0-21swapper/307:10:023
87599154,9cyclictest5911-21sed08:55:022
86799155,7cyclictest21-21ksoftirqd/108:05:001
86799151,5cyclictest16873-21cron07:55:011
86799151,11cyclictest6406-21irqcore08:55:151
86799151,11cyclictest27507-21idleruntime-cro08:25:011
86799151,11cyclictest22340-21sort08:10:021
85799159,4cyclictest11576-21awk07:40:010
85799158,5cyclictest1-21systemd07:49:120
85799152,10cyclictest10927-21munin-plugin-st09:05:010
85799151,3cyclictest0-21swapper/008:15:530
85799151,3cyclictest0-21swapper/007:52:050
85799151,3cyclictest0-21swapper/007:34:560
85799151,11cyclictest6459-21diskstats07:25:150
85799151,11cyclictest5902-21tr07:20:020
85799151,11cyclictest5885-21ls08:55:010
88399149,3cyclictest0-21swapper/308:02:163
86799141,9cyclictest11167-21latency_hist09:10:021
86799141,5cyclictest27233-21cron08:20:011
85799148,4cyclictest0-21swapper/008:05:310
88399138,3cyclictest0-21swapper/308:24:423
88399138,3cyclictest0-21swapper/307:45:463
88399131,9cyclictest0-21swapper/308:19:443
88399131,7cyclictest0-21swapper/308:55:023
86799131,9cyclictest886-21latency_hist07:10:011
88399126,4cyclictest0-21swapper/308:50:013
88399121,9cyclictest0-21swapper/307:50:123
88399121,7cyclictest0-21swapper/307:21:183
88399113,4cyclictest33-21ksoftirqd/309:01:453
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional