You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 13:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sun Apr 19, 2026 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
243299663,54cyclictest28440-21ssh21:50:110
243299556,45cyclictest23355-21ssh23:35:140
243299556,45cyclictest23355-21ssh23:35:130
243299533,38cyclictest12454-21ssh22:35:120
244099482,37cyclictest15808-21ssh00:25:141
243299481,35cyclictest3403-21ssh22:30:110
243299472,37cyclictest26854-21ssh22:25:110
243299471,38cyclictest30078-21ssh21:15:120
243299452,34cyclictest32092-21ssh22:10:110
243299450,40cyclictest14789-21apt-get20:30:130
243299450,40cyclictest1356-21ssh21:35:120
244099441,32cyclictest1346-21ssh21:35:131
2457994224,15cyclictest35966irq/58-eth023:40:123
243299421,36cyclictest21416-21apt-get22:40:100
2449994118,17cyclictest261rcuc/220:35:142
244099411,32cyclictest15658-21ssh21:25:131
243299412,35cyclictest2607-21apt-get19:10:120
243299412,35cyclictest2607-21apt-get19:10:120
243299401,32cyclictest0-21swapper/020:20:120
243299391,34cyclictest0-21swapper/000:00:130
243299391,29cyclictest0-21swapper/022:20:130
245799383,25cyclictest35966irq/58-eth021:10:143
244999381,17cyclictest1365-21ssh21:35:132
244099383,28cyclictest0-21swapper/122:45:131
243299383,22cyclictest30298-21ssh00:15:120
243299381,27cyclictest0-21swapper/021:40:130
243299381,25cyclictest4570-21sh23:24:370
243299380,35cyclictest19682-21apt-get20:35:150
243299380,34cyclictest0-21swapper/020:05:130
243299380,34cyclictest0-21swapper/000:35:130
243299380,24cyclictest4608-21ssh22:47:330
245799372,21cyclictest35966irq/58-eth022:00:133
244999372,29cyclictest3579-21ssh23:23:252
244999371,13cyclictest27-21ksoftirqd/223:10:142
2432993718,14cyclictest12-21ksoftirqd/000:30:130
243299371,32cyclictest1553-21apt-get23:05:120
243299371,29cyclictest10669-21apt-get23:10:130
243299371,27cyclictest15991-21ssh22:55:130
243299371,23cyclictest2472-21ntp_192.168.11419:40:230
243299370,23cyclictest20638-21diskmemload23:50:040
243299370,22cyclictest20638-21diskmemload21:29:400
243299370,21cyclictest12594-21ssh22:51:270
245799364,25cyclictest35966irq/58-eth023:35:133
245799364,25cyclictest35966irq/58-eth023:35:123
2449993623,8cyclictest27-21ksoftirqd/222:33:562
244999362,18cyclictest21580-21ssh23:33:272
244099361,24cyclictest28732-21ssh23:20:131
243299361,21cyclictest1038-21ssh23:57:470
243299361,20cyclictest31967-21ssh21:33:210
243299361,19cyclictest13558-21ssh00:22:470
2432993611,18cyclictest12-21ksoftirqd/021:00:110
243299360,31cyclictest0-21swapper/023:30:120
243299360,23cyclictest18329-21ssh00:06:540
243299360,22cyclictest20978-21sh00:25:540
243299351,30cyclictest0-21swapper/019:55:110
243299351,27cyclictest21327-21ssh00:10:120
243299351,20cyclictest8173-21rm23:44:410
243299350,19cyclictest13424-21ssh23:29:260
244099344,21cyclictest21-21ksoftirqd/123:30:131
244099343,24cyclictest5301-21sh23:25:141
244099343,23cyclictest24723-21ssh21:30:131
2440993425,3cyclictest20638-21diskmemload21:10:131
2440993422,6cyclictest0-21swapper/123:55:141
2440993419,5cyclictest21-21ksoftirqd/120:30:101
243299341,29cyclictest20448-21ntp_192.168.11419:25:250
243299341,19cyclictest29057-21ssh22:06:500
243299340,20cyclictest27831-21ssh21:49:450
243299340,19cyclictest11706-21ssh21:57:360
244999332,25cyclictest5749-21ssh22:11:562
244999332,24cyclictest5304-21ssh23:25:142
2449993318,9cyclictest0-21swapper/200:15:132
244999331,16cyclictest930-21snmpd19:55:132
2432993319,10cyclictest0-21swapper/020:15:110
243299331,29cyclictest11150-21cut22:15:210
243299331,29cyclictest0-21swapper/019:20:120
243299331,28cyclictest0-21swapper/019:15:120
243299330,28cyclictest0-21swapper/020:00:130
244999326,16cyclictest0-21swapper/222:25:122
2449993220,8cyclictest27346irq/40-dwc2_hso22:20:142
243299321,28cyclictest25018-21kthreadcore19:30:220
244999317,19cyclictest2423-21cyclictest23:19:562
244999312,19cyclictest27346irq/40-dwc2_hso21:30:122
2449993113,7cyclictest0-21swapper/221:20:142
244999311,18cyclictest12464-21ssh22:35:132
244099312,24cyclictest5118-21ssh21:55:141
243299311,27cyclictest4136-21kthreadcore20:50:220
243299311,27cyclictest30239-21swap19:35:280
243299311,26cyclictest32731-21ntpq20:45:250
243299311,26cyclictest29791-21latency_hist21:15:020
243299311,18cyclictest0-21swapper/022:03:320
243299310,27cyclictest16080-21apt-get21:05:140
245799303,18cyclictest35966irq/58-eth019:50:123
245799303,13cyclictest35966irq/58-eth000:25:143
244099304,19cyclictest201rcuc/123:10:131
244099301,22cyclictest12345-21ssh00:05:121
2432993018,5cyclictest0-21swapper/019:45:140
243299301,26cyclictest14700-21ssh23:46:500
243299301,25cyclictest9990-21/usr/sbin/munin20:25:230
243299301,24cyclictest0-21swapper/023:15:110
243299300,26cyclictest27983-21find20:10:100
243299300,26cyclictest10313-21ssh21:20:260
245799293,14cyclictest35966irq/58-eth022:30:113
245799293,13cyclictest0-21swapper/323:30:143
2457992916,6cyclictest0-21swapper/322:25:123
245799291,15cyclictest35966irq/58-eth023:25:133
244999295,19cyclictest1987-21ssh23:58:562
244999293,21cyclictest0-21swapper/222:43:172
244999292,21cyclictest0-21swapper/220:00:152
2449992913,11cyclictest27-21ksoftirqd/222:56:542
244099293,19cyclictest28469-21ssh21:50:131
244099293,19cyclictest20638-21diskmemload23:40:141
244099291,23cyclictest19439-21ssh21:45:141
2440992910,3cyclictest131rcu_preempt20:55:151
243299291,24cyclictest8714-21sort19:50:130
243299291,24cyclictest25175-21ssh23:00:150
243299291,24cyclictest25175-21ssh23:00:150
243299290,26cyclictest6528-21cpuspeed20:55:150
245799282,17cyclictest35966irq/58-eth019:30:133
2457992812,9cyclictest35966irq/58-eth023:50:133
244999283,16cyclictest27346irq/40-dwc2_hso20:40:122
2449992815,7cyclictest27-21ksoftirqd/220:30:102
244099285,13cyclictest23329-21apt-get23:35:111
244099285,13cyclictest23329-21apt-get23:35:111
244099282,19cyclictest1582-21rm23:05:131
243299280,26cyclictest29043-21sshd20:41:560
245799273,17cyclictest0-21swapper/321:25:543
245799273,16cyclictest0-21swapper/321:50:553
245799273,15cyclictest0-21swapper/323:23:563
244999276,4cyclictest27-21ksoftirqd/200:05:152
244999274,16cyclictest27346irq/40-dwc2_hso23:35:042
244999274,16cyclictest27346irq/40-dwc2_hso23:35:042
244999272,18cyclictest1559-21ssh23:05:132
244999271,15cyclictest19451-21ssh00:25:272
245799262,16cyclictest35966irq/58-eth023:10:133
244999263,15cyclictest27346irq/40-dwc2_hso20:50:122
244999262,17cyclictest24893-21ssh00:30:142
244099264,13cyclictest753-21sendmail-mta22:50:121
244099262,20cyclictest23106-21ssh22:05:151
243299261,22cyclictest2535-21cat19:10:010
245799258,12cyclictest0-21swapper/300:30:133
245799253,14cyclictest0-21swapper/322:15:153
244999258,12cyclictest27-21ksoftirqd/221:43:012
244999252,16cyclictest0-21swapper/221:55:052
244999252,14cyclictest0-21swapper/220:20:112
244999251,18cyclictest6863-21ssh00:20:132
244099254,17cyclictest0-21swapper/122:55:131
2457992412,8cyclictest35966irq/58-eth022:40:123
244999247,4cyclictest27-21ksoftirqd/222:05:292
244999245,7cyclictest27-21ksoftirqd/223:47:192
244999243,6cyclictest0-21swapper/221:45:542
244999242,16cyclictest27346irq/40-dwc2_hso20:05:152
244999241,8cyclictest20638-21diskmemload21:53:032
2449992414,7cyclictest27-21ksoftirqd/221:25:152
244999241,19cyclictest930-21snmpd21:15:132
244099243,15cyclictest8793-21ssh22:15:151
2440992414,6cyclictest211ksoftirqd/120:35:141
245799234,11cyclictest35966irq/58-eth022:35:143
245799233,15cyclictest0-21swapper/322:05:133
245799232,12cyclictest0-21swapper/322:50:123
245799231,17cyclictest36066irq/59-eth000:05:113
245799231,11cyclictest35966irq/58-eth021:40:123
244999233,12cyclictest27346irq/40-dwc2_hso19:15:132
244999232,6cyclictest0-21swapper/222:15:552
2449992312,9cyclictest28737-21ssh21:13:542
244999231,12cyclictest7008-21ssh22:50:132
2449992310,2cyclictest131rcu_preempt22:00:132
244999223,13cyclictest27346irq/40-dwc2_hso19:25:142
244999222,15cyclictest0-21swapper/222:49:552
2449992212,8cyclictest0-21swapper/219:40:252
244999221,17cyclictest27511-21kthreadcore19:35:192
2449992211,3cyclictest27-21ksoftirqd/220:25:242
244999221,12cyclictest4031-21fschecks_time19:45:162
2449992210,9cyclictest0-21swapper/200:35:592
244099222,16cyclictest21434-21ssh23:50:261
244099222,14cyclictest3393-21ssh22:30:101
244099221,18cyclictest21976-21uniq19:30:171
244099221,18cyclictest11994-21cron19:20:001
245799213,10cyclictest36066irq/59-eth022:10:113
245799212,2cyclictest0-21swapper/320:45:123
244999219,9cyclictest0-21swapper/200:10:132
244999215,10cyclictest27346irq/40-dwc2_hso19:20:132
244999214,5cyclictest27-21ksoftirqd/223:40:252
244999214,12cyclictest261rcuc/219:10:132
244999214,12cyclictest261rcuc/219:10:122
244999214,11cyclictest27346irq/40-dwc2_hso20:15:122
244999212,13cyclictest27346irq/40-dwc2_hso19:50:132
244999211,11cyclictest6587-21cut20:55:172
2449992110,9cyclictest30399-21ssh23:01:062
2449992110,9cyclictest30399-21ssh23:01:052
244099212,14cyclictest1371-21ssh00:35:131
2440992112,6cyclictest11390-21fschecks_count21:00:131
244099211,14cyclictest971-21latency_hist00:35:031
244099211,14cyclictest21427-21ssh22:40:121
245799209,9cyclictest0-21swapper/300:37:163
245799202,13cyclictest0-21swapper/322:45:143
245799201,12cyclictest0-21swapper/323:55:143
244999208,9cyclictest0-21swapper/200:00:122
244999203,8cyclictest31181-21kthreadcore20:10:202
244999201,9cyclictest15149-21grep21:00:262
244999201,8cyclictest640-21tail20:45:282
2449992011,7cyclictest0-21swapper/219:30:172
2449992010,8cyclictest0-21swapper/223:50:132
244099209,7cyclictest19790-21fschecks_time23:15:151
244099209,2cyclictest0-21swapper/100:00:241
244099208,9cyclictest29477-21cut20:45:151
244099208,3cyclictest131rcu_preempt00:20:191
244099204,9cyclictest2423-21cyclictest19:10:131
244099204,9cyclictest2423-21cyclictest19:10:121
244099203,14cyclictest16377-21kthreadcore19:55:201
244099201,9cyclictest15272-21awk21:25:011
244099201,15cyclictest14174-21ssh22:00:141
244099201,14cyclictest10381-21ssh21:40:131
245799193,11cyclictest0-21swapper/300:22:553
245799192,13cyclictest0-21swapper/321:45:543
2457991911,5cyclictest35966irq/58-eth022:20:133
245799191,13cyclictest0-21swapper/320:00:303
245799191,12cyclictest0-21swapper/321:15:233
2457991910,5cyclictest35966irq/58-eth021:30:133
244999191,9cyclictest930-21snmpd21:05:232
244099196,3cyclictest131rcu_preempt23:00:271
244099196,3cyclictest131rcu_preempt23:00:271
2440991912,4cyclictest21893-21irqrtprio00:10:191
2440991912,4cyclictest0-21swapper/120:05:291
244099191,14cyclictest21526-21ntp_192.168.11422:20:251
244099191,10cyclictest1247-21tr00:15:201
245799188,6cyclictest0-21swapper/300:18:313
245799183,9cyclictest0-21swapper/322:55:223
245799182,11cyclictest0-21swapper/321:35:553
245799182,10cyclictest0-21swapper/321:20:153
245799182,10cyclictest0-21swapper/320:10:133
245799182,10cyclictest0-21swapper/320:05:153
245799182,10cyclictest0-21swapper/319:20:143
245799182,10cyclictest0-21swapper/319:15:153
245799181,9cyclictest0-21swapper/323:15:123
2457991811,5cyclictest35966irq/58-eth021:58:413
245799181,14cyclictest0-21swapper/323:45:293
245799181,11cyclictest35966irq/58-eth000:10:133
245799181,10cyclictest0-21swapper/323:05:123
2457991810,5cyclictest0-21swapper/323:00:333
2457991810,5cyclictest0-21swapper/323:00:323
244999180,14cyclictest2575-21latency_hist19:10:022
244099189,6cyclictest4947-21cat20:50:221
244099182,5cyclictest0-21swapper/122:29:561
244099182,13cyclictest11678-21kthreadcore20:25:191
244099181,9cyclictest7973-21cut20:20:191
244099181,9cyclictest11452-21tr19:50:211
244099181,14cyclictest32106-21ssh22:10:121
244099181,14cyclictest30780-21ssh21:15:181
244099181,10cyclictest25158-21kthreadcore20:40:181
245799179,6cyclictest0-21swapper/300:04:413
245799172,9cyclictest0-21swapper/320:50:133
245799172,9cyclictest0-21swapper/320:40:143
245799172,9cyclictest0-21swapper/320:35:043
245799172,9cyclictest0-21swapper/320:30:133
245799172,10cyclictest0-21swapper/320:15:133
245799172,10cyclictest0-21swapper/319:40:143
245799171,12cyclictest0-21swapper/320:00:013
245799171,10cyclictest0-21swapper/320:55:153
244099172,12cyclictest22150-21timerwakeupswit20:00:301
244099172,11cyclictest2530-21ntp_states19:40:261
244099172,11cyclictest0-21swapper/121:05:141
2440991711,4cyclictest12564-21df22:35:161
244099171,12cyclictest28092-21diskstats20:10:141
244099171,12cyclictest11508-21tr23:45:191
245799162,9cyclictest0-21swapper/321:00:123
245799161,10cyclictest0-21swapper/320:25:273
244099164,9cyclictest16102-21unixbench_multi19:20:281
244099163,10cyclictest4568-21kthreadcore19:45:191
244099162,11cyclictest31198-21cron19:40:001
244099161,6cyclictest18926-21kthreadcore19:25:201
245799159,4cyclictest33-21ksoftirqd/319:25:183
245799151,9cyclictest0-21swapper/319:10:163
245799151,9cyclictest0-21swapper/319:10:163
244099151,11cyclictest397-21sort20:15:131
245799149,3cyclictest0-21swapper/321:05:233
244099141,10cyclictest2476-21cat19:10:011
245799130,9cyclictest0-21swapper/319:35:143
245799122,6cyclictest0-21swapper/320:20:133
245799111,7cyclictest0-21swapper/319:10:013
245799111,5cyclictest0-21swapper/319:45:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional