You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-27 - 06:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon Jul 27, 2026 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16037994637,6cyclictest35966irq/58-eth000:25:163
16037994636,7cyclictest35966irq/58-eth023:50:123
16037994635,7cyclictest35966irq/58-eth021:40:143
16037994631,11cyclictest35966irq/58-eth000:00:153
16037994631,11cyclictest35966irq/58-eth000:00:143
1602199391,32cyclictest12064-21ssh00:15:141
1601399397,22cyclictest12-21ksoftirqd/021:35:150
1602199383,26cyclictest15531-21ssh23:05:131
1603799375,24cyclictest35966irq/58-eth019:15:153
1603799363,28cyclictest35966irq/58-eth019:10:123
1603799363,20cyclictest35966irq/58-eth020:25:123
16037993619,10cyclictest35966irq/58-eth023:45:143
1602199352,3cyclictest131rcu_preempt22:00:121
1603799343,21cyclictest35966irq/58-eth021:20:153
1602999341,24cyclictest22742-21ssh23:45:142
1602199340,26cyclictest8175-21ssh23:55:131
16037993317,9cyclictest35966irq/58-eth022:25:133
1602999321,28cyclictest3121-21rm00:10:172
1602999321,22cyclictest13707-21sh22:10:122
1602999321,22cyclictest13707-21sh22:10:122
1603799304,14cyclictest35966irq/58-eth022:20:143
1602999301,21cyclictest10186-21ssh23:20:142
1601399304,16cyclictest6587-21ssh00:30:140
1602999293,22cyclictest261rcuc/220:40:122
1602999291,15cyclictest1866-21diskmemload22:15:142
16029992911,15cyclictest27-21ksoftirqd/200:05:142
1603799289,12cyclictest35966irq/58-eth023:10:133
1603799285,13cyclictest35966irq/58-eth021:30:123
16037992812,10cyclictest35966irq/58-eth022:10:133
16037992812,10cyclictest35966irq/58-eth022:10:133
1603799278,10cyclictest0-21swapper/321:10:143
1603799271,15cyclictest35966irq/58-eth021:45:133
1602199275,13cyclictest21-21ksoftirqd/123:10:111
1602199272,19cyclictest6292-21ssh21:46:071
16021992719,5cyclictest0-21swapper/123:25:141
16021992719,5cyclictest0-21swapper/123:25:141
1603799264,15cyclictest0-21swapper/323:40:153
1603799264,13cyclictest0-21swapper/322:30:163
1603799261,17cyclictest0-21swapper/300:10:163
1602999262,20cyclictest15555-21ssh23:05:152
1602999262,15cyclictest17363-21ssh22:30:142
16029992610,5cyclictest131rcu_preempt22:40:122
16029992610,12cyclictest19654-21sshd19:10:262
1602199262,19cyclictest28025-21ntp_states21:40:271
1602199261,20cyclictest11443-21ssh21:15:161
1601399262,15cyclictest11437-21sh21:15:160
1603799253,19cyclictest35966irq/58-eth021:15:163
1603799253,17cyclictest0-21swapper/322:05:123
1603799253,16cyclictest35966irq/58-eth023:20:153
16037992513,8cyclictest0-21swapper/321:35:223
1602999252,19cyclictest27346irq/40-dwc2_hso23:30:142
1602999252,18cyclictest20472-21ssh21:20:162
16029992520,3cyclictest27346irq/40-dwc2_hso21:30:112
16029992512,10cyclictest3258-21kthreadcore21:45:222
1602999251,20cyclictest26377-21ssh22:35:162
16021992511,9cyclictest28221-21df23:30:151
1601399252,18cyclictest31744-21apt-get22:20:120
1603799243,14cyclictest0-21swapper/322:35:163
1603799243,13cyclictest35966irq/58-eth020:10:143
1603799243,13cyclictest0-21swapper/321:55:133
1603799243,12cyclictest0-21swapper/322:40:153
1603799242,15cyclictest0-21swapper/300:30:153
1602999243,17cyclictest27346irq/40-dwc2_hso19:30:132
1602999243,17cyclictest27346irq/40-dwc2_hso00:00:152
1602999243,17cyclictest27346irq/40-dwc2_hso00:00:142
1602999243,14cyclictest27346irq/40-dwc2_hso20:35:142
1602999242,14cyclictest27346irq/40-dwc2_hso19:35:152
16029992418,4cyclictest20071-21ssh21:35:342
16029992416,6cyclictest1990-21ssh21:10:022
16029992413,8cyclictest0-21swapper/223:40:202
16029992411,5cyclictest12119-21sshd23:01:232
1602199249,3cyclictest1866-21diskmemload23:15:451
1602199247,4cyclictest131rcu_preempt21:35:221
16021992415,5cyclictest0-21swapper/121:50:131
1601399248,3cyclictest131rcu_preempt22:00:160
1601399245,12cyclictest17353-21ssh22:30:140
1601399242,16cyclictest19180-21rm23:25:160
1601399242,16cyclictest19180-21rm23:25:150
16013992410,3cyclictest12-21ksoftirqd/019:45:150
1603799234,13cyclictest0-21swapper/300:35:143
1603799233,13cyclictest0-21swapper/322:45:143
1603799232,13cyclictest35966irq/58-eth023:55:123
1602999239,11cyclictest28381-21hwlatdetect22:00:172
1602999235,11cyclictest27346irq/40-dwc2_hso00:20:132
1602999233,15cyclictest0-21swapper/220:00:122
1602999233,14cyclictest27346irq/40-dwc2_hso19:20:132
1602999232,16cyclictest19185-21ssh21:55:152
1602999232,13cyclictest0-21swapper/220:25:122
16029992317,3cyclictest2285-21ssh21:10:152
1602999231,13cyclictest930-21snmpd22:05:122
16021992310,4cyclictest1866-21diskmemload22:18:081
16013992315,6cyclictest0-21swapper/023:20:280
1603799225,10cyclictest0-21swapper/323:00:143
1603799223,12cyclictest0-21swapper/322:15:143
1603799222,16cyclictest35966irq/58-eth023:35:163
1602999224,15cyclictest27346irq/40-dwc2_hso20:30:122
1602999224,13cyclictest27-21ksoftirqd/220:45:112
1602999222,18cyclictest0-21swapper/221:54:262
16029992211,9cyclictest3455-21ssh21:27:232
1602999221,18cyclictest1686-21sed22:55:312
16029992211,8cyclictest0-21swapper/219:25:282
1602999221,17cyclictest8505-21grep22:25:172
1602999221,16cyclictest27346irq/40-dwc2_hso23:10:112
1602999221,11cyclictest1686-21sh22:20:212
16029992210,8cyclictest27-21ksoftirqd/200:25:142
1602199221,8cyclictest7491-21cat22:40:301
16021992215,4cyclictest0-21swapper/123:35:161
16021992212,3cyclictest1866-21diskmemload22:05:121
1602199221,15cyclictest15523-21sh00:35:131
1601399222,15cyclictest1107-21apt-get23:15:120
1601399221,17cyclictest22824-21ls23:45:150
1603799219,10cyclictest0-21swapper/322:55:223
1603799212,15cyclictest0-21swapper/323:30:143
16037992110,8cyclictest0-21swapper/323:07:393
1602999218,4cyclictest20574-21cron22:50:012
1602999218,10cyclictest4694-21ps20:05:192
1602999214,14cyclictest27-21ksoftirqd/220:50:132
1602999214,10cyclictest27346irq/40-dwc2_hso21:00:112
1602999213,9cyclictest15471-21/usr/sbin/munin00:35:232
1602999213,14cyclictest27346irq/40-dwc2_hso23:50:122
1602999212,14cyclictest27346irq/40-dwc2_hso20:20:122
1602999212,11cyclictest15476-21ls21:15:322
1602999212,10cyclictest21383-21kthreadcore20:55:192
1602999212,10cyclictest0-21swapper/222:50:122
1602999212,10cyclictest0-21swapper/222:50:122
1602999211,17cyclictest6745-21fschecks_count00:30:182
1602999211,16cyclictest0-21swapper/219:45:152
1602999211,12cyclictest24258-21kthreadcore19:50:202
1602999211,11cyclictest14137-21diskstats20:15:162
1602999211,10cyclictest8892-21head23:35:312
1602999211,10cyclictest2989-21kthreadcore23:15:222
1602199219,9cyclictest21-21ksoftirqd/123:50:011
1602199211,5cyclictest0-21swapper/121:30:011
16021992113,5cyclictest0-21swapper/119:20:141
1602199211,2cyclictest131rcu_preempt22:35:311
1601399217,3cyclictest131rcu_preempt20:55:020
1601399211,16cyclictest23681-21apt-get20:25:110
1601399211,15cyclictest15522-21ssh00:35:140
1601399211,14cyclictest17421-21ntp_states22:10:270
1601399211,14cyclictest17421-21ntp_states22:10:260
1603799207,9cyclictest35966irq/58-eth023:15:143
1603799203,11cyclictest0-21swapper/323:25:143
1603799203,11cyclictest0-21swapper/323:25:133
1602999205,12cyclictest27346irq/40-dwc2_hso21:40:132
1602999202,9cyclictest11159-21kthreadcore23:55:232
1602999202,15cyclictest12495-21latency20:10:232
1602999201,11cyclictest14984-21ssh00:15:232
1602999201,10cyclictest22749-21ntp_192.168.11423:25:272
1602999201,10cyclictest22749-21ntp_192.168.11423:25:262
1602999201,10cyclictest14625-21grep19:40:212
1602199206,3cyclictest131rcu_preempt00:00:281
1602199206,3cyclictest131rcu_preempt00:00:271
1602199201,4cyclictest21-21ksoftirqd/121:20:291
16021992011,6cyclictest24902-21grep19:15:311
1602199201,15cyclictest13697-21apt-get22:10:131
1602199201,15cyclictest13697-21apt-get22:10:121
1602199201,14cyclictest13787-21ssh23:40:161
16021992010,3cyclictest1866-21diskmemload00:26:041
1602199200,5cyclictest4223-21cut21:10:211
16013992012,6cyclictest2885-21sshd23:50:250
16013992010,7cyclictest9572-21cron21:50:010
1601399200,7cyclictest5241-21ps22:05:170
1603799193,11cyclictest0-21swapper/322:01:113
1603799193,11cyclictest0-21swapper/321:53:273
1603799193,11cyclictest0-21swapper/300:23:263
1603799191,12cyclictest0-21swapper/319:40:223
16037991910,6cyclictest35966irq/58-eth000:05:153
1602999191,9cyclictest27259-21fschecks_count19:55:152
1602999191,8cyclictest22923-21kthreadcore19:15:212
1602199196,8cyclictest21-21ksoftirqd/119:55:131
1602199192,4cyclictest131rcu_preempt22:50:201
1602199192,4cyclictest131rcu_preempt22:50:191
1602199191,5cyclictest22892-21kthreadcore20:55:221
16021991912,5cyclictest27174-21sshd20:25:271
16021991911,6cyclictest13731-21ssh23:20:241
1602199191,12cyclictest22131-21ls20:20:211
1602199191,10cyclictest1866-21diskmemload23:50:471
1602199191,10cyclictest0-21swapper/123:04:391
1602199191,10cyclictest0-21swapper/122:55:071
16021991910,6cyclictest7523-21cat00:10:331
1601399199,8cyclictest31691-21users19:55:310
1601399199,7cyclictest8468-21df_inode22:25:150
1601399198,3cyclictest131rcu_preempt00:16:460
1601399192,7cyclictest12-21ksoftirqd/020:55:110
1601399192,14cyclictest3679-21irqrtprio19:30:180
1601399192,12cyclictest2050-21ssh22:39:260
1601399192,12cyclictest18592-21apt-get21:55:020
1601399191,9cyclictest0-21swapper/022:19:290
1601399191,6cyclictest1866-21diskmemload00:21:160
1601399191,6cyclictest11641-21/usr/sbin/munin23:55:240
16013991913,3cyclictest12-21ksoftirqd/022:55:250
16013991912,4cyclictest24754-21hwlatdetect23:10:160
16013991911,6cyclictest1090-21ssh21:25:310
1601399191,15cyclictest2950-21apt-get22:40:120
16013991911,5cyclictest0-21swapper/019:15:030
1603799189,6cyclictest0-21swapper/321:25:183
1603799182,10cyclictest0-21swapper/320:40:143
1603799182,10cyclictest0-21swapper/319:30:133
1603799181,10cyclictest0-21swapper/300:20:023
1602999181,9cyclictest16073-21latency_hist19:10:022
1602199189,7cyclictest31871-21df_inode22:20:151
1602199189,6cyclictest2994-21sed19:30:021
1602199187,9cyclictest16646-21tr20:50:201
1602199186,9cyclictest1254-21tr20:35:191
1602199182,10cyclictest31976-21apt-get20:00:131
1602199181,9cyclictest28490-21cut00:05:221
1602199181,9cyclictest1866-21diskmemload21:32:261
1602199181,8cyclictest0-21swapper/122:28:481
1602199181,8cyclictest0-21swapper/100:21:301
1602199181,2cyclictest0-21swapper/122:45:441
16021991812,4cyclictest0-21swapper/119:30:171
16021991811,5cyclictest1863-21cron21:10:021
1602199181,15cyclictest18099-21kthreadcore22:30:201
1602199181,14cyclictest6875-21tr20:05:221
16021991810,6cyclictest9061-21cut00:30:231
1601399189,6cyclictest8568-21sshd00:11:200
1601399181,9cyclictest11020-21latency_hist21:15:010
1601399181,5cyclictest15091-21munin-run20:50:010
16013991812,4cyclictest0-21swapper/000:25:140
16013991811,4cyclictest0-21swapper/019:30:020
1601399180,8cyclictest18597-21munin-plugin-st21:55:020
1601399180,12cyclictest24160-21ls23:10:010
1603799179,5cyclictest35966irq/58-eth022:50:123
1603799179,5cyclictest35966irq/58-eth022:50:113
1603799172,9cyclictest0-21swapper/321:00:113
1603799172,9cyclictest0-21swapper/320:50:153
1603799172,10cyclictest0-21swapper/319:20:123
16037991712,3cyclictest33-21ksoftirqd/319:45:293
16037991711,4cyclictest33-21ksoftirqd/319:35:303
1603799171,12cyclictest0-21swapper/320:00:283
1603799171,10cyclictest0-21swapper/319:50:143
1602199179,6cyclictest877-21lldpd20:15:141
1602199179,6cyclictest877-21lldpd19:35:271
1602199176,3cyclictest1866-21diskmemload21:58:441
1602199171,13cyclictest8947-21apt-get20:10:031
1601399179,5cyclictest0-21swapper/019:15:130
1601399178,7cyclictest121ksoftirqd/019:50:120
1601399176,8cyclictest6928-21tail23:00:170
1601399172,12cyclictest4848-21ssh23:35:180
1601399172,11cyclictest26124-21latency_hist00:05:020
1601399172,11cyclictest26124-21latency_hist00:05:020
1601399171,8cyclictest0-21swapper/000:06:330
1601399171,7cyclictest24812-21irqcore21:40:180
1601399171,6cyclictest12076-21diskstats22:45:150
1601399171,3cyclictest0-21swapper/020:25:020
1601399171,14cyclictest28118-21sh23:30:130
16013991711,4cyclictest0-21swapper/023:40:170
16013991711,4cyclictest0-21swapper/020:05:260
1601399171,13cyclictest21152-21ssh21:20:210
1601399171,11cyclictest12603-21ssh21:32:250
1601399170,7cyclictest30503-21kthreadcore21:05:200
1603799162,9cyclictest0-21swapper/320:35:143
1603799161,13cyclictest0-21swapper/320:10:013
1603799161,12cyclictest0-21swapper/319:30:023
1603799161,10cyclictest0-21swapper/320:30:233
1602199167,7cyclictest26842-21kthreadcore21:00:181
1602199161,4cyclictest690-21munin-run20:35:011
1602199161,12cyclictest22366-21apt-get19:50:131
1602199161,10cyclictest13092-21sed19:40:171
1601399168,6cyclictest3537-21cut20:35:210
1601399162,11cyclictest14525-21kthreadcore19:40:200
1601399161,6cyclictest18644-21munin-run20:20:010
1601399161,6cyclictest13809-21cron20:15:010
1601399161,6cyclictest10310-21sed20:45:010
1601399161,12cyclictest20972-21ssh22:50:120
1601399161,12cyclictest20972-21ssh22:50:110
16013991610,4cyclictest0-21swapper/020:30:150
1603799151,11cyclictest0-21swapper/321:10:023
16037991510,3cyclictest0-21swapper/320:19:253
1602199157,6cyclictest20437-21cat19:10:301
1602199151,11cyclictest5747-21apt-get20:40:121
1602199151,11cyclictest17558-21apt-get19:45:121
1602199151,11cyclictest13485-21kthreadcore20:45:211
1601399159,4cyclictest0-21swapper/020:00:180
1601399151,11cyclictest25905-21apt-get19:20:120
1601399151,11cyclictest24928-21apt-get21:00:120
1601399151,10cyclictest16052-21wc19:10:010
1603799149,3cyclictest0-21swapper/320:20:153
1602199141,10cyclictest15943-21latency_hist19:10:021
1601399141,11cyclictest7947-21apt-get19:35:130
1603799132,8cyclictest0-21swapper/320:45:123
1603799131,9cyclictest0-21swapper/320:55:283
1603799131,3cyclictest0-21swapper/319:55:183
1603799111,7cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional