You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-09 - 21:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sat May 09, 2026 12:45:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8312998045,26cyclictest35966irq/58-eth011:05:133
829699723,56cyclictest3696-21ssh12:15:111
828899703,7cyclictest131rcu_preempt07:45:120
8312996357,4cyclictest35966irq/58-eth011:20:123
8305996323,5cyclictest16517-21ssh11:10:112
8305996323,5cyclictest16517-21ssh11:10:102
828899625,44cyclictest12659-21ssh09:20:140
828899542,41cyclictest5588-21ssh10:10:150
8288994630,7cyclictest12-21ksoftirqd/011:05:110
829699458,5cyclictest131rcu_preempt10:20:131
8288994525,6cyclictest12-21ksoftirqd/008:10:130
830599431,34cyclictest20126-21ssh11:30:122
831299424,26cyclictest35966irq/58-eth009:55:143
830599421,19cyclictest3577-21ssh09:15:152
831299393,23cyclictest35966irq/58-eth010:45:123
831299392,21cyclictest35966irq/58-eth011:35:123
830599392,23cyclictest12669-21ssh12:20:092
829699399,18cyclictest21-21ksoftirqd/107:20:121
828899384,24cyclictest12684-21ssh12:20:100
830599366,18cyclictest27-21ksoftirqd/207:50:122
829699361,26cyclictest0-21swapper/111:40:141
828899364,27cyclictest9259-21ssh10:30:120
828899364,20cyclictest30689-21ssh12:30:140
830599352,16cyclictest29085-21ssh11:35:132
830599351,6cyclictest131rcu_preempt12:15:122
8288993519,5cyclictest12-21ksoftirqd/009:40:140
831299342,18cyclictest35966irq/58-eth012:20:103
8296993420,8cyclictest32637-21sh11:55:131
828899342,29cyclictest30988-21ssh11:00:120
828899341,27cyclictest21726-21ssh09:25:130
8312993312,11cyclictest0-21swapper/309:45:133
829699334,22cyclictest7203-21ssh12:35:141
829699333,22cyclictest18279-21ssh10:35:131
8296993313,8cyclictest8276-21cyclictest07:55:141
828899336,19cyclictest151rcuc/009:35:140
828899331,27cyclictest14607-21ssh10:15:140
831299321,16cyclictest35966irq/58-eth010:50:123
830599325,21cyclictest8276-21cyclictest10:05:132
830599325,21cyclictest8276-21cyclictest10:05:122
830599321,17cyclictest27171-21ssh12:10:132
8296993223,4cyclictest5561-21apt-get10:10:151
828899323,17cyclictest20116-21ssh11:30:130
831299315,16cyclictest35966irq/58-eth011:00:113
8296993122,4cyclictest27743-21apt-get07:30:131
8296993118,8cyclictest12659-21apt-get12:20:101
8296993116,8cyclictest0-21swapper/109:25:151
828899312,24cyclictest21727-21ssh12:25:140
828899312,21cyclictest19915-21ssh10:00:120
831299308,10cyclictest35966irq/58-eth011:40:133
831299304,15cyclictest0-21swapper/310:15:133
8312993015,6cyclictest0-21swapper/308:55:113
830599305,15cyclictest27346irq/40-dwc2_hso07:40:132
830599305,15cyclictest27346irq/40-dwc2_hso07:40:122
830599303,16cyclictest27346irq/40-dwc2_hso07:25:122
830599302,18cyclictest27346irq/40-dwc2_hso08:45:122
831299297,13cyclictest0-21swapper/309:30:133
831299297,11cyclictest35966irq/58-eth011:15:123
831299294,14cyclictest35966irq/58-eth012:10:123
831299291,14cyclictest35966irq/58-eth012:25:113
830599293,14cyclictest27346irq/40-dwc2_hso09:45:132
830599293,11cyclictest0-21swapper/207:35:132
830599292,20cyclictest0-21swapper/209:05:152
830599292,15cyclictest14571-21ssh11:45:132
830599291,21cyclictest7555-21ssh11:05:122
830599283,16cyclictest27346irq/40-dwc2_hso08:00:122
830599282,20cyclictest0-21swapper/208:50:142
829699289,15cyclictest0-21swapper/112:00:141
829699285,19cyclictest3948-21ssh10:45:121
829699282,18cyclictest18146-21sh12:05:141
829699282,18cyclictest18146-21sh12:05:141
831299270,18cyclictest35966irq/58-eth011:55:123
830599272,19cyclictest0-21swapper/208:35:162
830599272,19cyclictest0-21swapper/208:35:162
830599271,20cyclictest32629-21ssh11:55:142
829699273,19cyclictest30680-21sh12:30:141
8288992720,3cyclictest121ksoftirqd/009:15:130
831299262,15cyclictest35966irq/58-eth010:35:133
830599263,13cyclictest27346irq/40-dwc2_hso08:05:152
830599262,15cyclictest27346irq/40-dwc2_hso11:20:112
830599262,10cyclictest7184-21ssh12:35:132
8296992618,4cyclictest20677-21apt-get08:30:131
828899268,12cyclictest12-21ksoftirqd/008:40:120
831299259,9cyclictest35966irq/58-eth011:25:123
831299253,13cyclictest35966irq/58-eth010:10:143
8312992521,2cyclictest0-21swapper/307:55:133
831299251,17cyclictest35966irq/58-eth010:05:133
831299251,17cyclictest35966irq/58-eth010:05:123
830599254,17cyclictest27346irq/40-dwc2_hso10:15:132
830599253,16cyclictest27346irq/40-dwc2_hso09:35:132
830599253,16cyclictest0-21swapper/210:35:142
830599252,17cyclictest0-21swapper/211:15:152
830599252,13cyclictest30690-21diskmemload12:30:142
830599251,15cyclictest6042-21latency_hist08:15:012
829699258,10cyclictest0-21swapper/107:36:261
829699254,4cyclictest21-21ksoftirqd/110:33:501
8296992518,4cyclictest28972-21apt-get10:05:131
8296992518,4cyclictest28972-21apt-get10:05:121
8296992514,8cyclictest19895-21apt-get10:00:111
828899251,20cyclictest151rcuc/009:50:130
831299244,12cyclictest0-21swapper/312:15:123
830599243,14cyclictest27346irq/40-dwc2_hso07:45:132
830599242,19cyclictest27357-21ssh10:40:122
830599242,13cyclictest27346irq/40-dwc2_hso07:15:122
830599241,9cyclictest5641-21ssh11:40:122
8305992416,5cyclictest25055-21ssh10:55:232
830599241,11cyclictest10947-21ssh09:55:152
8296992417,4cyclictest7245-21apt-get09:35:121
8296992416,3cyclictest22897-21apt-get07:25:141
829699241,5cyclictest21-21ksoftirqd/111:50:321
828899244,8cyclictest12-21ksoftirqd/010:55:120
828899243,14cyclictest0-21swapper/011:45:110
828899241,18cyclictest27046-21rm09:10:140
828899241,18cyclictest12892-21find10:50:110
828899241,17cyclictest23653-21ssh11:50:150
830599232,15cyclictest27346irq/40-dwc2_hso09:50:112
8305992312,9cyclictest17638-21ssh10:34:442
8305992311,8cyclictest0-21swapper/209:40:132
829699238,8cyclictest21-21ksoftirqd/109:11:031
829699233,4cyclictest21-21ksoftirqd/110:15:221
829699232,4cyclictest21-21ksoftirqd/111:35:271
8296992317,4cyclictest201rcuc/109:30:141
829699231,6cyclictest21-21ksoftirqd/109:45:121
8296992316,3cyclictest1886-21apt-get09:50:131
8296992315,4cyclictest10920-21apt-get09:55:141
829699231,4cyclictest21-21ksoftirqd/110:40:261
8296992314,4cyclictest0-21swapper/110:25:151
829699231,15cyclictest16536-21ssh11:10:121
829699231,15cyclictest16536-21ssh11:10:111
828899232,17cyclictest27620-21ls08:00:230
828899231,18cyclictest25788-21ls11:15:170
828899231,18cyclictest23718-21cut10:20:150
831299225,10cyclictest0-21swapper/309:35:123
830599229,11cyclictest614-21dbus-daemon07:30:092
830599224,8cyclictest25864-21kthreadcore10:20:212
830599223,15cyclictest27346irq/40-dwc2_hso10:10:142
830599222,11cyclictest27346irq/40-dwc2_hso08:40:112
830599221,9cyclictest21743-21grep09:25:152
8305992211,9cyclictest28464-21ssh12:27:482
830599221,15cyclictest27346irq/40-dwc2_hso12:05:152
830599221,15cyclictest27346irq/40-dwc2_hso12:05:142
830599221,10cyclictest21237-21apt-get09:25:012
830599220,15cyclictest21698-21sed10:55:012
830599220,10cyclictest4461-21usage_Anybus10:25:322
829699226,8cyclictest21-21ksoftirqd/109:19:131
829699224,4cyclictest21-21ksoftirqd/108:45:281
829699224,3cyclictest131rcu_preempt12:27:261
829699223,5cyclictest21-21ksoftirqd/111:08:301
829699223,4cyclictest21-21ksoftirqd/111:45:181
829699223,4cyclictest21-21ksoftirqd/111:02:121
8296992217,3cyclictest201rcuc/109:43:121
8296992216,4cyclictest201rcuc/109:24:431
8296992215,4cyclictest30369-21apt-get08:40:131
829699221,4cyclictest21-21ksoftirqd/108:25:281
8296992214,4cyclictest844-21munin-node11:30:141
829699221,16cyclictest12925-21ssh10:50:111
829699221,16cyclictest0-21swapper/109:00:131
828899221,9cyclictest27214-21grep12:10:130
8288992214,5cyclictest32591-21/usr/sbin/munin11:55:260
828899221,17cyclictest10946-21diskmemload09:55:150
828899221,16cyclictest23942-21ls08:30:220
831299212,12cyclictest0-21swapper/310:30:143
8312992113,4cyclictest0-21swapper/311:10:123
8312992113,4cyclictest0-21swapper/311:10:113
8312992111,6cyclictest0-21swapper/310:00:123
831299211,12cyclictest35966irq/58-eth011:45:123
830599218,10cyclictest2660-21unixbench_singl11:00:302
830599212,11cyclictest27984-21usage_Anybus11:50:312
829699218,11cyclictest0-21swapper/111:25:121
829699213,4cyclictest21-21ksoftirqd/111:20:181
829699213,4cyclictest21-21ksoftirqd/108:55:221
829699211,5cyclictest21-21ksoftirqd/109:05:231
8296992115,3cyclictest8455-21apt-get07:10:121
829699211,11cyclictest29058-21acpi08:05:131
829699211,10cyclictest0-21swapper/112:13:101
829699210,5cyclictest91-21pr/ttyS007:20:021
8288992111,7cyclictest7519-21awk10:45:220
828899211,15cyclictest10824-21sshd12:35:250
8288992110,8cyclictest12774-21ntpq12:00:260
831299207,10cyclictest35966irq/58-eth012:30:143
831299203,12cyclictest0-21swapper/310:55:123
8312992014,4cyclictest0-21swapper/312:10:013
8312992014,4cyclictest0-21swapper/312:10:003
830599205,6cyclictest10855-21sendmail08:20:012
830599203,9cyclictest22207-21kthreadcore08:30:192
830599203,9cyclictest11084-21/usr/sbin/munin11:25:262
830599203,11cyclictest27346irq/40-dwc2_hso10:00:102
830599202,9cyclictest18352-21kthreadcore09:00:192
830599202,15cyclictest0-21swapper/207:10:132
830599201,9cyclictest14129-21kthreadcore08:55:192
830599201,8cyclictest988066ssh12:00:192
830599201,16cyclictest27543-21cut09:10:182
829699205,3cyclictest131rcu_preempt11:15:281
829699202,5cyclictest21-21ksoftirqd/108:35:221
829699202,5cyclictest21-21ksoftirqd/108:35:221
829699201,6cyclictest23049-21grep10:55:201
8296992013,3cyclictest1461-21apt-get08:10:121
828899209,7cyclictest3782-21sshd10:25:260
828899207,3cyclictest131rcu_preempt11:10:170
828899207,3cyclictest131rcu_preempt11:10:170
828899202,13cyclictest27328-21ssh10:40:120
828899202,10cyclictest12-21ksoftirqd/007:55:130
828899201,14cyclictest30633-21ssh09:30:140
828899200,15cyclictest1516-21latency_hist09:50:020
831299192,11cyclictest0-21swapper/310:25:163
831299192,11cyclictest0-21swapper/309:05:143
831299192,11cyclictest0-21swapper/307:15:143
831299192,10cyclictest0-21swapper/307:35:143
8312991913,4cyclictest0-21swapper/310:40:263
8312991913,4cyclictest0-21swapper/309:50:213
831299191,15cyclictest0-21swapper/307:25:173
831299191,13cyclictest0-21swapper/308:15:233
831299191,12cyclictest0-21swapper/308:20:233
831299191,12cyclictest0-21swapper/308:20:233
831299191,12cyclictest0-21swapper/308:10:283
831299191,11cyclictest0-21swapper/308:05:153
8312991910,6cyclictest35966irq/58-eth009:20:143
8312991910,6cyclictest0-21swapper/309:44:343
830599191,7cyclictest30617-21ssh09:30:122
8305991913,3cyclictest26631-21diskmemload10:45:172
830599191,15cyclictest19499-21grep07:55:172
830599191,15cyclictest19152-21kthreadcore07:20:192
830599190,9cyclictest18897-21kthreadcore08:25:242
829699199,8cyclictest9737-21apt-get07:45:121
829699192,4cyclictest21-21ksoftirqd/108:15:281
829699190,7cyclictest5070-21fschecks_count07:40:171
829699190,7cyclictest5070-21fschecks_count07:40:161
828899197,3cyclictest131rcu_preempt11:40:330
828899192,4cyclictest131rcu_preempt12:05:200
828899192,4cyclictest131rcu_preempt12:05:200
828899191,9cyclictest0-21swapper/011:29:250
8288991911,6cyclictest16930-21cron09:00:010
8288991911,5cyclictest3931-21fschecks_time12:15:160
828899191,15cyclictest26042-21irqrtprio08:35:200
828899191,15cyclictest26042-21irqrtprio08:35:190
828899191,14cyclictest0-21swapper/011:35:130
831299189,5cyclictest35966irq/58-eth009:10:143
831299188,4cyclictest36066irq/59-eth011:50:153
831299185,6cyclictest0-21swapper/311:30:133
831299182,10cyclictest0-21swapper/308:50:133
831299182,10cyclictest0-21swapper/308:35:053
831299182,10cyclictest0-21swapper/308:35:043
831299182,10cyclictest0-21swapper/308:30:143
831299181,12cyclictest0-21swapper/307:55:023
831299181,11cyclictest0-21swapper/307:30:143
831299181,10cyclictest0-21swapper/308:45:133
8312991810,6cyclictest0-21swapper/309:27:303
8312991810,6cyclictest0-21swapper/309:20:013
8312991810,3cyclictest0-21swapper/312:00:173
830599181,9cyclictest11244-21cut08:20:172
830599181,9cyclictest11244-21cut08:20:162
830599181,14cyclictest8321-21latency_hist07:10:012
829699185,4cyclictest21-21ksoftirqd/107:50:251
829699184,3cyclictest131rcu_preempt08:50:201
829699181,14cyclictest11659-21irqrtprio08:20:191
829699181,14cyclictest11659-21irqrtprio08:20:181
8296991810,6cyclictest24265-21apt-get08:00:131
828899188,7cyclictest22144-21diskstats09:05:170
828899188,7cyclictest17828-21cron07:20:010
828899187,8cyclictest3049-21kthreadcore11:20:190
828899185,10cyclictest20285-21kthreadcore10:35:230
828899183,13cyclictest11413-21grep08:50:270
828899182,13cyclictest672-21grep10:05:300
828899182,13cyclictest672-21grep10:05:290
828899182,11cyclictest0-21swapper/007:30:140
828899181,15cyclictest13057-21latency_hist07:15:020
828899181,10cyclictest9479-21users07:40:320
828899181,10cyclictest9479-21users07:40:310
828899180,6cyclictest12100-21kthreadcore08:20:200
828899180,6cyclictest12100-21kthreadcore08:20:190
831299179,6cyclictest0-21swapper/310:22:283
831299172,9cyclictest0-21swapper/307:45:133
831299172,10cyclictest0-21swapper/308:00:143
831299171,9cyclictest35966irq/58-eth012:35:123
831299171,12cyclictest0-21swapper/308:25:173
829699172,4cyclictest21-21ksoftirqd/107:10:021
828899175,7cyclictest0-21swapper/008:25:150
828899172,10cyclictest0-21swapper/007:35:140
8288991712,3cyclictest32620-21sendmail_mailqu08:05:280
828899171,12cyclictest14644-21diskstats07:50:160
8288991710,4cyclictest21372-21cat07:20:220
831299162,10cyclictest0-21swapper/307:40:153
831299162,10cyclictest0-21swapper/307:40:143
831299162,10cyclictest0-21swapper/307:10:143
828899163,10cyclictest27428-21munin-run07:30:010
828899162,10cyclictest0-21swapper/008:45:130
828899161,12cyclictest7361-21tr08:15:190
828899151,11cyclictest17374-21grep09:00:150
831299142,9cyclictest0-21swapper/308:40:123
831299141,11cyclictest0-21swapper/307:10:023
828899141,10cyclictest8313-21latency_hist07:10:010
831299131,8cyclictest0-21swapper/307:20:193
831299122,6cyclictest0-21swapper/309:00:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional