You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 10:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Thu May 07, 2026 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6928998249,24cyclictest35966irq/58-eth000:25:123
691799605,44cyclictest0-21swapper/219:40:122
6928995842,11cyclictest35966irq/58-eth000:05:133
691799582,29cyclictest24970-21diskmemload21:55:122
690099572,42cyclictest2961-21ssh00:15:120
690099546,45cyclictest18439-21ssh22:00:130
692899514,41cyclictest35966irq/58-eth000:30:113
6928995024,18cyclictest35966irq/58-eth023:50:123
692899484,28cyclictest35966irq/58-eth021:50:133
692899472,31cyclictest35966irq/58-eth020:10:123
6917994421,17cyclictest7829-21apt-get22:30:122
6917994416,22cyclictest13588-21apt-get23:45:122
6917994416,22cyclictest13588-21apt-get23:45:112
691799431,25cyclictest4524-21ssh23:40:122
692899424,22cyclictest35966irq/58-eth000:10:123
692899423,25cyclictest35966irq/58-eth021:10:133
6928994221,14cyclictest35966irq/58-eth021:20:123
6917994215,22cyclictest27346irq/40-dwc2_hso20:05:142
690999425,32cyclictest24970-21diskmemload21:50:111
690999421,32cyclictest6221-21ssh23:05:141
692899419,23cyclictest0-21swapper/300:35:113
692899391,34cyclictest0-21swapper/300:20:123
6917993922,9cyclictest17964-21apt-get19:55:122
692899383,31cyclictest0-21swapper/322:50:123
691799382,29cyclictest9234-21ssh22:10:522
6917993814,19cyclictest0-21swapper/222:40:122
6917993720,3cyclictest15215-21apt-get23:10:122
690999371,33cyclictest20057-21sh21:25:131
6917993622,9cyclictest6211-21apt-get23:05:142
6917993615,14cyclictest27415-21apt-get22:05:102
691799361,15cyclictest6623-21ssh00:35:122
6917993522,2cyclictest22704-21apt-get23:50:122
6917993521,10cyclictest8372-21apt-get19:45:122
6917993518,11cyclictest2953-21apt-get00:15:122
690999353,20cyclictest7860-21ssh22:30:131
6909993530,3cyclictest536-21usb-storage19:15:121
692899346,18cyclictest0-21swapper/323:40:133
691799344,17cyclictest29087-21ssh21:30:122
6917993426,3cyclictest11946-21apt-get19:15:112
691799342,28cyclictest9985-21ssh23:25:162
692899334,16cyclictest35966irq/58-eth020:35:123
692899330,1cyclictest0-21swapper/300:00:123
691799334,15cyclictest0-21swapper/222:50:112
691799328,17cyclictest27346irq/40-dwc2_hso19:30:132
6917993214,13cyclictest0-21swapper/200:20:122
690999322,23cyclictest31297-21ssh22:41:081
692899313,24cyclictest0-21swapper/322:45:113
6928993114,9cyclictest0-21swapper/320:40:123
692899311,25cyclictest0-21swapper/323:05:143
692899310,2cyclictest0-21swapper/322:15:113
6917993117,10cyclictest27-21ksoftirqd/200:10:122
6928993013,11cyclictest0-21swapper/323:48:523
6928993013,11cyclictest0-21swapper/323:48:523
692899301,2cyclictest0-21swapper/323:20:143
692899301,2cyclictest0-21swapper/323:15:153
692899301,2cyclictest0-21swapper/323:15:143
692899301,24cyclictest0-21swapper/322:30:133
692899301,22cyclictest0-21swapper/322:40:123
691799303,20cyclictest0-21swapper/220:15:142
691799302,22cyclictest0-21swapper/222:15:532
691799302,14cyclictest14887-21fschecks_time21:40:162
691799302,14cyclictest14887-21fschecks_time21:40:152
6917993014,12cyclictest0-21swapper/221:45:122
6917993012,14cyclictest0-21swapper/219:20:112
691799301,18cyclictest27346irq/40-dwc2_hso20:25:132
690999304,23cyclictest13619-21ssh23:45:141
690999304,23cyclictest13619-21ssh23:45:131
692899291,24cyclictest0-21swapper/320:05:153
691799298,17cyclictest0-21swapper/223:00:122
691799293,21cyclictest0-21swapper/222:03:512
6917992917,8cyclictest7053-21apt-get19:10:132
6928992821,5cyclictest35966irq/58-eth023:25:133
6928992817,5cyclictest35966irq/58-eth023:00:133
692899281,24cyclictest0-21swapper/319:20:113
691799286,18cyclictest25297-21wc19:25:292
691799284,16cyclictest27346irq/40-dwc2_hso00:25:132
691799283,17cyclictest27346irq/40-dwc2_hso20:50:122
691799283,16cyclictest27346irq/40-dwc2_hso22:25:132
691799282,21cyclictest0-21swapper/223:31:522
690999287,8cyclictest21-21ksoftirqd/122:00:151
690099284,16cyclictest1978-21ssh21:15:120
690099282,20cyclictest1159-21ssh22:43:520
690099281,23cyclictest0-21swapper/023:55:130
692899271,23cyclictest0-21swapper/322:35:113
692899271,22cyclictest0-21swapper/321:15:123
691799277,16cyclictest0-21swapper/222:35:122
691799276,17cyclictest0-21swapper/222:55:142
691799276,12cyclictest0-21swapper/223:18:362
691799276,12cyclictest0-21swapper/223:18:352
691799272,15cyclictest27346irq/40-dwc2_hso21:35:122
691799272,10cyclictest0-21swapper/223:35:142
6917992713,9cyclictest0-21swapper/221:15:122
691799271,21cyclictest7919-21ssh22:46:102
690099272,9cyclictest0-21swapper/023:18:520
690099272,9cyclictest0-21swapper/023:18:520
6900992716,7cyclictest0-21swapper/019:45:120
692899263,18cyclictest0-21swapper/322:55:523
692899261,21cyclictest0-21swapper/319:55:133
692899261,21cyclictest0-21swapper/319:25:133
692899261,18cyclictest0-21swapper/323:35:113
692899260,23cyclictest0-21swapper/322:25:123
692899260,1cyclictest0-21swapper/322:05:103
691799264,14cyclictest27346irq/40-dwc2_hso20:30:122
691799263,14cyclictest27346irq/40-dwc2_hso21:00:142
691799262,20cyclictest0-21swapper/220:35:152
690999265,18cyclictest27413-21ssh22:05:101
690999262,19cyclictest26023-21ssh22:56:111
690999262,18cyclictest5695-21ssh23:04:511
690999262,18cyclictest29521-21ssh23:15:531
690999262,18cyclictest29521-21ssh23:15:521
690099262,19cyclictest4819-21ssh22:26:510
692899253,17cyclictest0-21swapper/300:15:133
692899252,18cyclictest0-21swapper/320:30:133
692899251,21cyclictest0-21swapper/322:20:243
692899251,21cyclictest0-21swapper/321:57:423
692899251,21cyclictest0-21swapper/321:40:113
692899251,21cyclictest0-21swapper/321:40:113
692899251,19cyclictest0-21swapper/323:10:113
692899250,2cyclictest0-21swapper/322:00:133
692899250,2cyclictest0-21swapper/321:45:153
691799256,15cyclictest20656-21ls21:05:172
691799254,13cyclictest27346irq/40-dwc2_hso20:00:112
691799253,8cyclictest17273-21ssh21:22:092
691799252,9cyclictest0-21swapper/220:40:112
691799251,21cyclictest2079-21ls19:35:232
691799250,12cyclictest1214-21awk00:30:232
690099251,3cyclictest131rcu_preempt00:00:130
6900992511,8cyclictest24970-21diskmemload00:27:570
692899245,15cyclictest35966irq/58-eth020:50:133
692899242,19cyclictest0-21swapper/321:25:303
692899241,2cyclictest0-21swapper/323:55:123
692899241,21cyclictest0-21swapper/319:10:123
692899241,20cyclictest0-21swapper/321:05:163
692899241,20cyclictest0-21swapper/320:45:273
692899241,19cyclictest0-21swapper/321:35:123
692899240,2cyclictest0-21swapper/320:55:123
692899240,20cyclictest0-21swapper/319:35:113
691799247,14cyclictest135-21systemd-journal00:00:012
691799244,16cyclictest0-21swapper/220:20:122
6917992417,4cyclictest958-21apt-get23:20:132
690099243,18cyclictest25587-21grep21:10:160
690099242,17cyclictest20070-21ssh21:25:130
690099242,16cyclictest6219-21sh23:05:130
692899233,17cyclictest0-21swapper/320:25:123
692899231,2cyclictest0-21swapper/323:30:143
692899231,19cyclictest0-21swapper/321:30:213
692899231,19cyclictest0-21swapper/321:00:193
692899231,17cyclictest0-21swapper/320:00:113
692899230,2cyclictest0-21swapper/319:15:173
6917992312,9cyclictest1580-21cron21:15:012
6917992311,9cyclictest17988-21kthreadcore00:05:192
691799231,18cyclictest24970-21diskmemload22:20:522
6917992310,7cyclictest15027-21ssh00:02:492
690999239,9cyclictest21-21ksoftirqd/122:20:121
690999233,17cyclictest26059-21latency_hist00:10:021
690999233,15cyclictest22733-21ssh23:50:131
690999232,17cyclictest10737-21apt-get20:55:121
690099232,17cyclictest21711-21fschecks_count19:25:170
690099232,15cyclictest24970-21diskmemload21:40:260
690099232,15cyclictest24970-21diskmemload21:40:250
690099231,18cyclictest15617-21cpu21:00:150
692899221,1cyclictest0-21swapper/319:45:293
692899221,19cyclictest0-21swapper/322:10:113
692899221,18cyclictest0-21swapper/320:20:273
692899221,18cyclictest0-21swapper/320:15:253
692899221,16cyclictest0-21swapper/319:30:133
691799221,14cyclictest0-21swapper/221:28:212
690999223,16cyclictest24970-21diskmemload23:55:151
690999223,16cyclictest1055-21latency00:30:231
690999222,15cyclictest4175-21ssh22:26:051
690999222,11cyclictest2952-21kthreadcore23:20:211
6909992211,8cyclictest8392-21df00:00:141
690999221,16cyclictest13283-21cut22:15:151
690099227,3cyclictest131rcu_preempt23:55:020
690099224,4cyclictest12-21ksoftirqd/023:10:270
690099222,17cyclictest28902-21grep20:40:140
690099222,17cyclictest27431-21kthreadcore19:30:190
690099222,17cyclictest22645-21ps20:30:260
690099221,8cyclictest14368-21usage_Anybus23:25:300
690099221,6cyclictest2647-21sshd21:31:530
6900992215,4cyclictest0-21swapper/021:20:120
690099221,18cyclictest992-21ssh23:20:130
692899211,17cyclictest0-21swapper/319:50:133
692899210,2cyclictest0-21swapper/319:40:113
691799213,9cyclictest1255-21kthreadcore20:10:182
691799213,11cyclictest0-21swapper/221:52:302
690999214,14cyclictest21099-21diskstats00:25:141
690999213,3cyclictest131rcu_preempt23:39:101
690999213,15cyclictest2475-21kthreadcore20:10:211
690999213,15cyclictest24543-21sed21:45:181
690999212,17cyclictest9604-21fschecks_time21:55:161
690999212,16cyclictest32501-21kthreadcore19:35:191
690999212,16cyclictest18532-21irqrtprio19:55:171
690999212,16cyclictest13158-21diskmemload21:20:201
690999211,5cyclictest4638-21df_inode23:40:151
6909992112,7cyclictest23695-21sshd20:30:501
690999211,17cyclictest7561-21irqrtprio19:10:171
690999211,17cyclictest20744-21latency_hist00:25:031
690999211,17cyclictest14687-21kthreadcore19:50:171
690999211,14cyclictest25588-21cut21:10:151
690099213,5cyclictest12-21ksoftirqd/023:00:170
690099213,15cyclictest11174-21kthreadcore20:20:190
690099212,4cyclictest131rcu_preempt00:30:110
690099212,16cyclictest27917-21tail21:45:300
690099212,16cyclictest27657-21timerandwakeup20:35:280
690099212,14cyclictest13007-21ssh22:30:520
690099211,5cyclictest21358-21munin-run19:25:010
690099211,17cyclictest27432-21ssh22:05:130
690099211,17cyclictest18492-21kthreadcore00:05:180
690099211,16cyclictest12293-21ssh22:50:190
690099211,15cyclictest10572-21ls00:35:280
690099211,14cyclictest7276-21timerandwakeup19:40:270
690099211,13cyclictest16547-21latency_hist19:20:010
690099211,10cyclictest0-21swapper/022:16:410
691799204,8cyclictest1970-21kthreadcore20:45:192
691799201,9cyclictest13369-21fschecks_time19:50:132
691799201,15cyclictest6982-21latency_hist19:10:012
690999209,8cyclictest21669-21df_inode19:25:151
690999203,14cyclictest17407-21irqrtprio19:20:171
690999202,16cyclictest3464-21cut00:15:171
690999202,16cyclictest29315-21fschecks_time21:30:141
690999202,15cyclictest7713-21/usr/sbin/munin22:10:251
690999202,15cyclictest28775-21cpu20:40:131
690999202,15cyclictest26579-21sed20:00:281
690999202,15cyclictest23409-21kthreadcore21:05:221
690999202,15cyclictest16027-21kthreadcore21:40:201
690999202,15cyclictest16027-21kthreadcore21:40:191
690999201,4cyclictest0-21swapper/123:26:231
690999201,4cyclictest0-21swapper/121:37:101
6909992014,4cyclictest0-21swapper/119:10:021
690999201,16cyclictest9888-21cut20:20:171
690999201,16cyclictest6868-21kthreadcore20:15:211
690999201,16cyclictest28085-21irqrtprio20:05:181
690099206,3cyclictest131rcu_preempt20:50:010
690099204,14cyclictest4767-21fschecks_time23:40:150
690099202,16cyclictest28220-21kernelversion20:05:200
690099202,16cyclictest21284-21kthreadcore23:30:200
690099202,16cyclictest18089-21fschecks_count19:55:160
690099202,15cyclictest4849-21fschecks_count20:15:170
690099202,15cyclictest22252-21apt-get23:50:010
690099202,15cyclictest22252-21apt-get23:50:010
690099202,15cyclictest14297-21kthreadcore19:50:160
690099201,17cyclictest21155-21kthreadcore21:05:190
690099201,16cyclictest27139-21kthreadcore00:10:180
690099201,10cyclictest0-21swapper/022:59:220
691799191,9cyclictest10930-21fschecks_time20:55:152
690999199,7cyclictest21047-21kthreadcore23:30:211
690999193,14cyclictest27422-21kthreadcore00:10:181
690999193,13cyclictest4496-21kthreadcore21:15:221
690999192,15cyclictest27603-21switchtime20:35:271
690999192,14cyclictest10965-21kthreadcore19:45:201
690999192,13cyclictest15236-21ssh23:10:111
690999191,15cyclictest4285-21kthreadcore19:40:191
690099192,15cyclictest12143-21tune2fs00:20:160
690099192,14cyclictest23325-21irqrtprio20:00:170
690099192,14cyclictest11938-21kthreadcore20:55:190
690099192,13cyclictest19078-21latency_hist20:30:020
690099191,16cyclictest31369-21fschecks_time19:35:160
690099191,16cyclictest2607-21kthreadcore20:10:200
690099191,16cyclictest12615-21cut21:55:230
690099191,15cyclictest6952-21latency_hist19:10:010
690999183,13cyclictest1666-21cut20:45:171
690999183,12cyclictest7018-21grep00:35:161
690999182,13cyclictest17869-21wc20:25:261
690999181,15cyclictest28257-21kthreadcore19:30:211
690999181,15cyclictest17063-21kthreadcore21:00:201
690999181,15cyclictest13735-21kthreadcore22:50:191
6909991810,5cyclictest17257-21irqcore22:35:161
690099183,13cyclictest6754-21cut20:50:180
690099183,13cyclictest14303-21awk21:40:000
690099182,14cyclictest4084-21ssh21:50:260
690099182,11cyclictest22039-21ssh22:35:520
690099181,9cyclictest0-21swapper/022:22:380
690099181,5cyclictest0-21swapper/022:11:420
690099181,14cyclictest9436-21kthreadcore19:10:200
6900991811,4cyclictest28009-21ls23:35:130
692899170,14cyclictest35966irq/58-eth019:10:023
690999172,12cyclictest6280-21cut20:50:171
6909991710,3cyclictest5301-21cut22:45:201
690099171,10cyclictest7042-21cat22:45:280
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional