You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-15 - 12:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Fri May 15, 2026 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9749997747,16cyclictest35966irq/58-eth021:10:143
974399551,30cyclictest13769-21ssh21:20:132
974399533,30cyclictest17589-21ssh23:10:152
973699502,37cyclictest26575-21ssh23:15:110
9739994830,7cyclictest21-21ksoftirqd/122:45:141
9749994536,6cyclictest35966irq/58-eth023:25:123
974999451,3cyclictest0-21swapper/300:05:173
974399453,25cyclictest24632-21ssh22:20:102
973699456,32cyclictest16023-21ssh23:45:130
974999446,23cyclictest35966irq/58-eth021:25:143
974999442,25cyclictest35966irq/58-eth019:20:113
974399441,26cyclictest31745-21sh21:30:132
973999442,35cyclictest11943-21sh21:55:121
973699444,35cyclictest8548-21sh23:05:130
973699442,34cyclictest28535-21ssh00:10:130
9743994330,8cyclictest27-21ksoftirqd/223:55:142
973699435,35cyclictest12293-21ssh23:25:130
973699435,28cyclictest31761-21ssh21:30:130
9749994234,5cyclictest35966irq/58-eth021:45:133
974999422,27cyclictest35966irq/58-eth020:00:123
974999422,25cyclictest35966irq/58-eth023:05:133
973999423,28cyclictest2971-21ssh21:50:151
973999421,33cyclictest13784-21ssh21:20:141
9739994128,6cyclictest27921-21diskmemload23:00:141
974999401,3cyclictest0-21swapper/321:50:143
974999401,2cyclictest0-21swapper/323:20:133
974999392,32cyclictest0-21swapper/321:55:113
9749993922,9cyclictest35966irq/58-eth023:30:143
974999392,22cyclictest35966irq/58-eth022:15:113
974399395,22cyclictest30302-21ssh23:35:142
974399391,28cyclictest0-21swapper/221:50:142
974399373,6cyclictest131rcu_preempt22:45:142
9743993721,10cyclictest27921-21diskmemload00:25:152
9736993716,16cyclictest26400-21ssh21:45:150
973699371,28cyclictest10323-21ssh22:30:140
973699371,28cyclictest10323-21ssh22:30:130
974999362,19cyclictest35966irq/58-eth019:25:103
973699363,9cyclictest12-21ksoftirqd/019:50:120
974999352,27cyclictest0-21swapper/321:00:133
974999351,2cyclictest0-21swapper/321:30:133
974999351,19cyclictest35966irq/58-eth023:35:143
974399351,29cyclictest15286-21diskmemload23:07:462
9739993516,10cyclictest9732-21cyclictest21:40:121
9739993422,9cyclictest21-21ksoftirqd/100:34:441
9736993425,4cyclictest0-21swapper/020:00:120
974999332,2cyclictest0-21swapper/323:00:143
974999332,26cyclictest0-21swapper/323:10:163
974999332,20cyclictest35966irq/58-eth022:45:133
974999332,17cyclictest35966irq/58-eth000:30:133
974399331,16cyclictest20583-21ssh22:52:422
973699336,17cyclictest20992-21ssh22:00:120
973699333,25cyclictest28325-21ssh21:10:140
974999325,16cyclictest35966irq/58-eth000:15:123
974999322,25cyclictest35966irq/58-eth022:10:143
974999322,14cyclictest35966irq/58-eth000:10:123
974999321,4cyclictest0-21swapper/322:55:143
974999321,27cyclictest0-21swapper/320:05:143
974399324,16cyclictest27346irq/40-dwc2_hso20:50:132
973999322,23cyclictest27963-21ssh22:55:441
9736993222,6cyclictest12-21ksoftirqd/023:35:130
9736993216,12cyclictest12-21ksoftirqd/023:23:440
974999311,2cyclictest0-21swapper/323:55:143
9743993111,13cyclictest21311-21ssh23:30:142
9739993121,6cyclictest131rcu_preempt20:25:111
973699311,20cyclictest1253-21ssh22:25:110
974999301,2cyclictest0-21swapper/322:25:093
974999301,2cyclictest0-21swapper/320:45:123
974999301,16cyclictest35966irq/58-eth022:40:123
974399302,11cyclictest6241-21ssh23:02:432
973999302,22cyclictest19778-21ssh23:28:441
9739993020,6cyclictest0-21swapper/121:05:121
974999292,4cyclictest0-21swapper/321:15:163
974999291,4cyclictest0-21swapper/322:50:143
974999291,3cyclictest0-21swapper/323:45:143
974999291,24cyclictest35966irq/58-eth000:20:143
974399294,15cyclictest27346irq/40-dwc2_hso21:00:132
974399292,21cyclictest4789-21ssh21:15:152
974399292,20cyclictest3637-21ssh00:13:452
973999291,19cyclictest24996-21ssh23:50:141
973699293,20cyclictest11940-21ssh21:55:120
973699292,19cyclictest29630-21sh22:57:450
973699292,18cyclictest21290-21sh23:30:130
973699292,15cyclictest19315-21ssh22:35:130
974999282,18cyclictest35966irq/58-eth020:55:133
974399283,17cyclictest27346irq/40-dwc2_hso22:25:112
974399283,15cyclictest27346irq/40-dwc2_hso21:45:142
974399283,14cyclictest27346irq/40-dwc2_hso20:55:122
974399282,20cyclictest0-21swapper/200:30:142
974399282,19cyclictest0-21swapper/220:10:132
974399281,20cyclictest22921-21ssh22:55:132
973999283,4cyclictest131rcu_preempt23:40:121
973999282,4cyclictest20981-21apt-get22:00:121
973999282,20cyclictest14209-21ssh00:00:241
973699283,7cyclictest131rcu_preempt00:16:450
973699282,20cyclictest7918-21ssh23:04:440
9736992817,4cyclictest15523-21ssh22:15:110
974999273,6cyclictest0-21swapper/321:35:143
974999273,18cyclictest35966irq/58-eth021:05:123
974999272,2cyclictest0-21swapper/300:35:163
974999271,21cyclictest35966irq/58-eth020:15:113
974399272,19cyclictest0-21swapper/200:35:162
974399272,18cyclictest27346irq/40-dwc2_hso19:45:132
9743992720,4cyclictest17360-21rm23:45:192
9743992716,7cyclictest27-21ksoftirqd/219:20:102
973999273,19cyclictest5102-21ssh00:15:141
973699271,22cyclictest22791-21ssh21:25:130
974999262,18cyclictest0-21swapper/321:40:133
974999262,15cyclictest0-21swapper/300:25:153
974999261,3cyclictest0-21swapper/320:20:123
974999261,2cyclictest0-21swapper/322:35:153
974999261,2cyclictest0-21swapper/322:30:133
974999261,2cyclictest0-21swapper/322:30:123
974999261,2cyclictest0-21swapper/320:30:123
974399265,14cyclictest27346irq/40-dwc2_hso19:40:122
974399264,15cyclictest27346irq/40-dwc2_hso19:25:122
974399263,18cyclictest27346irq/40-dwc2_hso23:40:132
974399262,9cyclictest922-21ssh22:40:452
973999265,12cyclictest131rcu_preempt20:20:131
973999264,11cyclictest131rcu_preempt23:20:131
973999261,21cyclictest22801-21ssh21:25:131
973999261,19cyclictest23463-21ssh23:11:441
973699265,3cyclictest131rcu_preempt00:36:170
9736992614,5cyclictest0-21swapper/020:10:130
973699261,17cyclictest32128-21ssh00:30:120
9736992610,4cyclictest12-21ksoftirqd/021:23:340
974999252,2cyclictest0-21swapper/322:00:123
974999252,17cyclictest0-21swapper/300:00:103
974999251,4cyclictest0-21swapper/320:40:123
974999251,3cyclictest0-21swapper/323:20:013
974999251,3cyclictest0-21swapper/322:20:103
974999251,2cyclictest0-21swapper/322:06:393
974399255,13cyclictest27346irq/40-dwc2_hso19:50:132
9743992513,8cyclictest17289-21/usr/sbin/munin21:40:222
9739992513,8cyclictest17174-21cron23:10:021
9739992513,6cyclictest0-21swapper/120:05:151
973699257,11cyclictest12-21ksoftirqd/020:25:120
973699252,4cyclictest12-21ksoftirqd/022:15:000
973699252,18cyclictest8329-21ssh21:35:150
973699252,18cyclictest21214-21ssh23:10:270
974999241,5cyclictest0-21swapper/319:40:133
974999241,4cyclictest0-21swapper/320:50:133
974999241,3cyclictest0-21swapper/321:20:083
974999241,3cyclictest0-21swapper/319:55:263
974999241,3cyclictest0-21swapper/319:50:283
974999241,2cyclictest0-21swapper/323:50:143
974999241,2cyclictest0-21swapper/319:15:243
974999241,20cyclictest0-21swapper/319:45:163
9749992410,9cyclictest35966irq/58-eth023:40:143
974399242,7cyclictest0-21swapper/220:20:032
974399241,8cyclictest27921-21diskmemload23:27:442
9739992414,7cyclictest13356-21cron21:20:011
973999241,18cyclictest15596-21ssh00:38:001
973699244,5cyclictest12-21ksoftirqd/022:45:290
974999231,19cyclictest0-21swapper/320:25:243
974999231,18cyclictest0-21swapper/319:40:013
974399232,7cyclictest0-21swapper/222:00:122
974399232,7cyclictest0-21swapper/221:05:112
974399232,6cyclictest0-21swapper/222:39:452
974399232,6cyclictest0-21swapper/222:05:142
974399232,16cyclictest0-21swapper/200:07:542
974399232,13cyclictest27346irq/40-dwc2_hso20:45:122
9743992313,6cyclictest0-21swapper/200:15:322
9743992312,9cyclictest28302-21ssh21:26:202
974399231,17cyclictest24995-21ssh23:50:142
9743992311,6cyclictest23569-21ssh22:19:192
9743992310,8cyclictest10559-21/usr/sbin/munin00:00:242
973999239,4cyclictest131rcu_preempt19:10:191
973999238,4cyclictest131rcu_preempt23:35:191
973999233,15cyclictest28317-21ssh22:40:141
973999232,17cyclictest30070-21ssh22:05:141
973999231,12cyclictest0-21swapper/120:50:201
973699239,8cyclictest12-21ksoftirqd/023:50:150
973699237,8cyclictest12-21ksoftirqd/021:53:320
973699236,9cyclictest12-21ksoftirqd/022:50:460
9736992315,6cyclictest151rcuc/019:45:260
9736992312,3cyclictest131rcu_preempt20:40:290
973699231,19cyclictest10256-21ls19:10:170
973699231,17cyclictest18149-21ls19:15:220
9736992311,4cyclictest12-21ksoftirqd/000:03:350
974999221,3cyclictest0-21swapper/320:35:173
974999221,3cyclictest0-21swapper/319:10:203
974399226,13cyclictest12315-21cut20:20:012
974399223,15cyclictest27346irq/40-dwc2_hso21:10:152
974399222,6cyclictest0-21swapper/200:22:452
9743992211,9cyclictest16228-21ssh21:39:122
973999229,3cyclictest131rcu_preempt23:15:291
973999228,4cyclictest29268-21kernelversion00:10:191
973999221,3cyclictest131rcu_preempt23:30:131
973699228,8cyclictest12-21ksoftirqd/000:24:550
973699228,2cyclictest131rcu_preempt21:17:460
973699227,8cyclictest12-21ksoftirqd/000:26:290
973699226,8cyclictest12-21ksoftirqd/000:07:510
973699226,3cyclictest131rcu_preempt23:55:310
973699225,3cyclictest131rcu_preempt23:40:290
9736992215,4cyclictest536-21usb-storage22:20:050
9736992213,4cyclictest0-21swapper/019:25:130
973699221,15cyclictest28307-21ssh22:40:140
974999211,2cyclictest0-21swapper/320:10:133
974999211,2cyclictest0-21swapper/319:30:153
974999211,2cyclictest0-21swapper/319:30:153
974399219,9cyclictest1419-21idleruntime-cro19:35:012
974399219,9cyclictest1419-21idleruntime-cro19:35:012
974399214,10cyclictest12974-21kthreadcore19:10:202
974399213,12cyclictest0-21swapper/223:20:122
974399212,16cyclictest27884-21kthreadcore20:00:202
974399212,11cyclictest3740-21kthreadcore19:35:202
974399211,5cyclictest0-21swapper/219:55:122
9743992111,8cyclictest0-21swapper/219:15:172
974399211,16cyclictest30773-21grep20:05:192
974399211,11cyclictest930-21snmpd20:40:172
974399211,11cyclictest30142-21grep23:15:232
974399211,10cyclictest10298-21ssh22:30:122
974399211,10cyclictest10298-21ssh22:30:122
973999217,9cyclictest21883-21kworker/1:123:47:451
9739992110,2cyclictest21-21ksoftirqd/100:20:201
973699217,8cyclictest12-21ksoftirqd/021:40:280
973699216,11cyclictest131rcu_preempt21:00:230
973699210,18cyclictest22424-21ls20:30:160
974999201,6cyclictest0-21swapper/319:10:013
974399205,6cyclictest18195-21kthreadcore20:25:182
974399202,9cyclictest6612-21ssh22:10:142
974399201,9cyclictest28834-21ps20:35:212
974399201,9cyclictest22341-21cut20:30:152
974399201,10cyclictest12654-21irqstats21:55:192
973999208,9cyclictest18399-21cat22:50:301
973999208,2cyclictest0-21swapper/123:59:151
973999205,8cyclictest131rcu_preempt19:20:181
973999202,10cyclictest1389-21kworker/1:019:35:201
973999201,8cyclictest26841-21latency_hist20:35:021
9739992014,4cyclictest0-21swapper/119:55:261
9739992012,6cyclictest21-21ksoftirqd/122:35:121
973999201,14cyclictest2841-21apt-get20:10:111
973699204,5cyclictest12-21ksoftirqd/020:20:270
973699203,4cyclictest12-21ksoftirqd/020:55:150
9736992014,4cyclictest0-21swapper/020:45:120
973699201,16cyclictest23369-21apt-get21:05:110
973699201,15cyclictest5389-21timerwakeupswit19:35:270
9736992010,4cyclictest12-21ksoftirqd/022:05:250
974399191,10cyclictest9785-21cat19:10:022
973999197,9cyclictest28368-21kthreadcore20:35:211
973999196,4cyclictest0-21swapper/122:30:011
973999192,12cyclictest19617-21ssh00:05:171
973999191,9cyclictest27063-21cat00:25:281
973999191,9cyclictest131rcu_preempt22:10:161
973999191,9cyclictest12154-21tr22:30:211
973999191,9cyclictest12154-21tr22:30:211
973999191,15cyclictest18791-21kthreadcore22:15:231
973999191,14cyclictest27921-21diskmemload21:35:141
973999191,10cyclictest26379-21ssh21:45:131
973999190,11cyclictest3853-21ssh21:30:321
973999190,11cyclictest1133-21tr20:40:201
973699198,5cyclictest131rcu_preempt19:55:160
973699196,3cyclictest131rcu_preempt19:40:210
9736991912,4cyclictest0-21swapper/020:50:130
973999181,9cyclictest30943-21kthreadcore19:30:191
973999181,9cyclictest30943-21kthreadcore19:30:191
973999181,5cyclictest8278-21cat20:45:291
9739991812,4cyclictest0-21swapper/119:25:261
9739991810,6cyclictest14873-21switchtime19:45:261
973999180,14cyclictest32161-21switchtime21:10:301
973699187,3cyclictest131rcu_preempt19:20:200
9736991812,4cyclictest0-21swapper/020:05:180
973999178,2cyclictest131rcu_preempt21:00:191
9739991711,4cyclictest9669-21latency_hist19:10:011
9739991711,4cyclictest10311-21unixbench_singl19:40:281
9739991710,4cyclictest0-21swapper/122:20:121
973999170,9cyclictest17368-21tr19:50:191
9736991711,4cyclictest0-21swapper/020:15:100
973699171,12cyclictest29552-21kthreadcore20:35:220
973999168,6cyclictest18303-21sshd20:57:441
973999163,11cyclictest21-21ksoftirqd/120:15:111
973999161,11cyclictest16098-21kthreadcore19:15:191
973999160,3cyclictest131rcu_preempt20:00:121
973699162,4cyclictest12-21ksoftirqd/019:10:020
973699162,10cyclictest0-21swapper/019:30:130
973699162,10cyclictest0-21swapper/019:30:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional