You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-05 - 23:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Wed Aug 05, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1179899627,46cyclictest10653-21ssh10:20:130
11817996127,29cyclictest27-21ksoftirqd/211:20:132
1181799563,36cyclictest29891-21ssh09:55:132
1180799531,43cyclictest5402-21ssh10:35:151
1180799474,9cyclictest21-21ksoftirqd/110:40:131
1181799452,17cyclictest25668-21sh11:40:132
11829994317,14cyclictest35966irq/58-eth012:15:153
1180799431,38cyclictest29882-21ssh09:55:131
1180799421,32cyclictest25669-21ssh11:40:131
11817994129,8cyclictest27-21ksoftirqd/212:30:142
1181799411,30cyclictest7523-21ssh11:30:132
1179899418,26cyclictest9098-21ssh10:55:130
1181799401,32cyclictest16647-21ssh11:35:132
1180799401,4cyclictest131rcu_preempt11:35:141
1179899382,32cyclictest9868-21sh12:25:120
1181799373,20cyclictest0-21swapper/212:13:462
11817993612,16cyclictest18152-21diskmemload11:00:142
1179899353,4cyclictest131rcu_preempt11:20:120
1179899343,27cyclictest15221-21ssh12:10:130
1179899341,29cyclictest18134-21ssh11:00:120
1179899340,7cyclictest845-21apt-get12:20:120
11829993318,9cyclictest35966irq/58-eth011:40:133
1181799338,9cyclictest9800-21kworker/2:111:15:152
1181799338,9cyclictest9800-21kworker/2:111:15:152
1181799334,10cyclictest27313-21ssh11:05:132
1179899333,26cyclictest18893-21ssh12:30:140
1179899332,27cyclictest6197-21ssh12:05:130
1182999329,15cyclictest0-21swapper/310:45:143
1182999321,17cyclictest35966irq/58-eth010:40:133
11807993220,9cyclictest21-21ksoftirqd/111:25:141
1180799321,5cyclictest131rcu_preempt10:30:141
1179899323,21cyclictest17660-21diskmemload10:05:150
1180799318,5cyclictest131rcu_preempt11:30:131
1180799314,18cyclictest21-21ksoftirqd/111:05:141
11798993124,4cyclictest151rcuc/011:45:140
1179899311,28cyclictest14409-21ssh10:40:130
1180799303,16cyclictest27919-21ssh12:35:131
1180799301,18cyclictest6487-21ssh10:00:131
1180799301,18cyclictest6487-21ssh10:00:131
11817992912,14cyclictest0-21swapper/210:40:132
1181799291,20cyclictest24258-21ssh12:15:142
1180799294,15cyclictest11368-21ssh11:50:121
1179899294,6cyclictest12-21ksoftirqd/010:00:140
1179899294,6cyclictest12-21ksoftirqd/010:00:130
1179899292,19cyclictest20222-21ssh11:18:320
1179899292,19cyclictest20222-21ssh11:18:320
1179899290,3cyclictest131rcu_preempt10:35:150
1182999283,10cyclictest35966irq/58-eth011:05:133
1181799282,20cyclictest0-21swapper/210:55:142
11817992817,7cyclictest27-21ksoftirqd/211:45:142
1181799281,15cyclictest20439-21ssh11:55:122
11807992821,4cyclictest15579-21sh10:05:161
11807992810,13cyclictest7926-1kworker/u9:110:58:301
1179899281,23cyclictest28762-21rm10:30:130
1181799272,20cyclictest0-21swapper/210:30:162
1181799272,19cyclictest856-21diskmemload12:20:132
1181799271,14cyclictest29545-21sh12:00:152
1181799271,11cyclictest7401-21ssh10:53:292
1180799272,21cyclictest9892-21ssh12:25:141
11807992712,4cyclictest0-21swapper/109:50:361
1182999262,16cyclictest35966irq/58-eth011:15:143
1182999262,16cyclictest35966irq/58-eth011:15:143
1182999262,15cyclictest0-21swapper/310:53:303
11829992616,6cyclictest35966irq/58-eth010:15:143
1181799264,15cyclictest27346irq/40-dwc2_hso10:20:132
1181799261,13cyclictest20836-21ssh09:50:102
1180799264,7cyclictest131rcu_preempt12:11:461
1180799262,19cyclictest24286-21ssh12:15:151
1180799261,18cyclictest23466-21rm10:45:131
11798992619,4cyclictest630-21Async09:36:310
1179899260,8cyclictest17271-21irqrtprio11:35:190
1181799254,15cyclictest27346irq/40-dwc2_hso10:00:132
1181799254,15cyclictest27346irq/40-dwc2_hso10:00:132
1181799252,8cyclictest5890-21ssh11:28:332
1180799259,10cyclictest21-21ksoftirqd/111:15:171
1180799259,10cyclictest21-21ksoftirqd/111:15:161
1180799252,18cyclictest857-21ssh12:20:131
1179899259,8cyclictest12-21ksoftirqd/009:42:030
1179899251,4cyclictest0-21swapper/010:19:470
11798992511,8cyclictest0-21swapper/010:45:070
11829992413,8cyclictest35966irq/58-eth009:50:103
1181799242,16cyclictest0-21swapper/212:35:462
1181799241,9cyclictest23459-21ssh10:45:122
11817992412,9cyclictest11721-21ssh11:50:162
1179899249,6cyclictest12-21ksoftirqd/009:10:590
1179899248,8cyclictest12-21ksoftirqd/012:02:210
1179899248,4cyclictest131rcu_preempt11:25:170
1179899245,5cyclictest12-21ksoftirqd/010:50:210
1179899244,6cyclictest12-21ksoftirqd/009:50:090
1179899244,10cyclictest131rcu_preempt08:50:020
1182999233,14cyclictest35966irq/58-eth010:55:123
11829992311,7cyclictest0-21swapper/312:25:143
1181799232,16cyclictest0-21swapper/212:08:412
11817992312,9cyclictest996-21ssh09:27:512
11817992312,9cyclictest9947-21ssh10:19:412
11817992312,3cyclictest9876-21ssh12:25:132
1181799231,11cyclictest14166-21ls10:40:022
1180799232,15cyclictest29548-21ssh12:00:151
11807992313,5cyclictest2440-21df11:45:151
1180799231,16cyclictest1204-21ssh10:15:141
1179899238,8cyclictest12-21ksoftirqd/011:11:140
1179899235,4cyclictest131rcu_preempt09:59:150
1179899235,3cyclictest131rcu_preempt08:55:010
1179899235,3cyclictest131rcu_preempt08:55:010
1179899232,4cyclictest131rcu_preempt11:05:150
1179899232,19cyclictest18986-21ssh11:53:460
11798992310,4cyclictest12-21ksoftirqd/012:37:060
1182999223,12cyclictest0-21swapper/311:25:463
11829992213,5cyclictest0-21swapper/311:50:293
1181799223,11cyclictest14751-21latency_hist08:15:022
1181799222,15cyclictest0-21swapper/210:25:132
11817992216,3cyclictest12656-21sh07:30:012
11817992211,9cyclictest10834-21ssh09:39:042
11817992210,9cyclictest1-21systemd07:18:002
11817992210,9cyclictest1-21systemd07:18:002
1180799227,4cyclictest21-21ksoftirqd/111:00:151
1180799221,13cyclictest14825-21cat08:15:031
1179899229,4cyclictest12-21ksoftirqd/009:48:350
1179899228,8cyclictest12-21ksoftirqd/010:12:000
1179899226,8cyclictest12-21ksoftirqd/011:55:330
1179899226,8cyclictest12-21ksoftirqd/009:19:460
1179899226,10cyclictest12-21ksoftirqd/011:40:200
1179899223,5cyclictest12-21ksoftirqd/009:33:110
11798992213,6cyclictest15062-21cron08:20:010
11798992212,4cyclictest0-21swapper/009:29:350
1179899221,16cyclictest19673-21ssh10:25:130
1182999219,8cyclictest35966irq/58-eth010:35:163
1182999213,12cyclictest0-21swapper/312:38:463
11829992114,4cyclictest33-21ksoftirqd/311:35:173
11829992110,7cyclictest35966irq/58-eth010:10:123
1181799219,9cyclictest26714-21ssh09:20:042
1181799217,11cyclictest6879-21kthreadcore11:10:212
1181799212,15cyclictest15310-21sh08:25:012
1181799212,11cyclictest16873-21munin-run08:55:012
1181799212,11cyclictest16873-21munin-run08:55:012
1181799211,14cyclictest28620-21ls10:10:282
11817992110,9cyclictest21245-21ssh09:14:012
1181799210,15cyclictest16269-21latency_hist09:45:022
1180799217,2cyclictest131rcu_preempt09:28:501
1180799215,3cyclictest131rcu_preempt11:55:201
1180799215,14cyclictest131rcu_preempt09:22:211
1180799213,9cyclictest0-21swapper/110:10:131
1180799211,15cyclictest0-21swapper/111:20:121
1179899216,3cyclictest131rcu_preempt08:55:040
1179899213,6cyclictest12-21ksoftirqd/012:15:150
1182999202,11cyclictest35966irq/58-eth011:55:123
1182999201,13cyclictest0-21swapper/311:30:143
11829992010,8cyclictest0-21swapper/309:39:183
1181799205,5cyclictest0-21swapper/208:05:012
1181799201,9cyclictest17549-21latency_hist09:05:012
1181799201,9cyclictest15721-21fschecks_count10:05:172
1181799201,15cyclictest16126-21latency_hist08:40:022
1181799201,14cyclictest17367-21latency_hist09:00:012
1181799201,10cyclictest17754-21latency_hist09:10:012
1181799201,10cyclictest15836-21latency_hist08:35:012
1181799201,10cyclictest12127-21latency_hist07:15:012
1180799203,2cyclictest131rcu_preempt10:50:261
1180799202,4cyclictest53646usb-storage08:35:101
1180799201,9cyclictest12878-21cron07:35:011
1180799201,13cyclictest10642-21sh10:20:121
11807992010,7cyclictest9851-21ntp_kernel_err12:05:261
1179899208,5cyclictest12-21ksoftirqd/009:20:370
1179899208,4cyclictest12-21ksoftirqd/008:35:010
1182999196,9cyclictest35966irq/58-eth011:00:133
1182999193,11cyclictest0-21swapper/312:34:463
1182999193,11cyclictest0-21swapper/311:22:473
1182999192,12cyclictest0-21swapper/312:13:413
1182999191,12cyclictest0-21swapper/310:10:013
11829991910,6cyclictest0-21swapper/309:31:093
1181799192,9cyclictest26679-21latency_hist09:20:022
1181799191,8cyclictest16378-21latency_hist08:45:022
1181799191,15cyclictest11898-21cat07:10:022
1181799191,13cyclictest7382-21latency_hist09:35:022
1181799191,10cyclictest16558-21latency_hist08:50:012
1180799197,4cyclictest17175-21cron09:00:011
1180799191,7cyclictest4633-21kthreadcore11:10:191
1180799191,3cyclictest131rcu_preempt09:11:271
11807991913,4cyclictest0-21swapper/112:30:191
1180799191,2cyclictest131rcu_preempt09:18:441
11807991910,6cyclictest19267-21cron10:25:021
1180799190,2cyclictest131rcu_preempt09:43:001
1180799190,2cyclictest131rcu_preempt09:37:241
1179899195,4cyclictest151rcuc/007:43:030
1179899195,4cyclictest151rcuc/007:43:030
11798991910,7cyclictest1-21systemd09:08:110
1182999189,6cyclictest35966irq/58-eth011:45:143
1182999188,8cyclictest0-21swapper/309:25:503
1182999187,8cyclictest0-21swapper/310:00:233
1182999187,8cyclictest0-21swapper/310:00:233
1182999183,11cyclictest0-21swapper/312:08:403
1182999182,9cyclictest0-21swapper/308:35:043
1182999181,9cyclictest0-21swapper/308:15:023
1182999181,13cyclictest0-21swapper/307:55:013
11829991810,6cyclictest0-21swapper/310:30:493
11829991810,6cyclictest0-21swapper/310:27:233
11829991810,6cyclictest0-21swapper/309:40:383
11829991810,6cyclictest0-21swapper/309:24:293
11829991810,6cyclictest0-21swapper/309:14:463
1181799187,8cyclictest14064-21awk08:00:012
1181799185,8cyclictest0-21swapper/208:10:012
1181799182,8cyclictest12451-21latency_hist07:25:012
1181799181,9cyclictest13836-21latency_hist07:55:012
1181799181,9cyclictest13267-21latency_hist07:40:012
1181799181,7cyclictest0-21swapper/209:49:232
1181799181,14cyclictest15538-21sh08:30:022
1181799181,14cyclictest15071-21munin-plugin-st08:20:012
1181799181,14cyclictest13537-21grep07:45:012
1181799181,14cyclictest13537-21grep07:45:012
1180799184,3cyclictest0-21swapper/109:47:401
1180799181,4cyclictest53646usb-storage08:50:101
1180799181,4cyclictest53646usb-storage08:50:101
1180799181,2cyclictest131rcu_preempt09:30:361
1180799181,14cyclictest16431-21ls08:45:021
1180799181,14cyclictest15512-21latency_hist08:25:021
1179899189,6cyclictest1-21systemd07:23:000
1179899184,3cyclictest131rcu_preempt09:35:480
1179899183,5cyclictest131rcu_preempt07:38:030
1179899182,5cyclictest53646usb-storage08:35:050
1179899182,11cyclictest14748-21apt-get08:15:020
1179899181,3cyclictest131rcu_preempt07:50:010
1179899181,3cyclictest131rcu_preempt07:30:030
11798991812,4cyclictest0-21swapper/008:05:010
1182999179,6cyclictest0-21swapper/310:21:183
1182999179,6cyclictest0-21swapper/309:56:593
1182999179,6cyclictest0-21swapper/309:45:133
1182999178,6cyclictest0-21swapper/311:10:533
1182999178,6cyclictest0-21swapper/309:16:233
1182999177,8cyclictest0-21swapper/312:03:533
1182999172,10cyclictest0-21swapper/312:20:133
1182999172,10cyclictest0-21swapper/308:55:043
1182999171,12cyclictest0-21swapper/307:25:013
1181799171,8cyclictest13581-21cron07:50:002
1181799171,13cyclictest12946-21latency_hist07:35:012
1180799176,4cyclictest131rcu_preempt08:05:021
1180799172,4cyclictest0-21swapper/107:30:011
1180799171,13cyclictest17654-21cron09:10:001
1180799170,13cyclictest14042-21idleruntime-cro08:00:011
1179899173,4cyclictest12-21ksoftirqd/007:20:020
1179899173,4cyclictest12-21ksoftirqd/007:20:010
1179899171,2cyclictest131rcu_preempt07:50:010
1182999161,12cyclictest0-21swapper/307:30:013
1180799163,9cyclictest15950-21cat08:35:021
1180799162,3cyclictest131rcu_preempt08:28:071
1180799162,10cyclictest13141-21sendmail-msp07:40:011
1180799160,6cyclictest12425-21idleruntime-cro07:25:001
1179899161,12cyclictest135-21systemd-journal09:05:010
1179899161,11cyclictest11945-21cron07:15:020
1182999151,8cyclictest0-21swapper/308:05:013
1180799156,2cyclictest131rcu_preempt07:42:341
1180799156,2cyclictest131rcu_preempt07:42:341
1180799151,7cyclictest0-21swapper/108:15:051
1180799151,11cyclictest16561-21awk08:50:011
1180799151,11cyclictest14525-21munin-run08:10:011
1180799151,10cyclictest12186-21sendmail07:20:011
1180799151,10cyclictest12186-21sendmail07:20:011
1180799150,2cyclictest131rcu_preempt07:55:021
1179899157,6cyclictest12-21ksoftirqd/008:25:010
1179899152,10cyclictest16370-21latency_hist08:45:020
1179899151,11cyclictest11899-21grep07:10:020
1179899151,10cyclictest14158-21latency_hist08:00:020
1179899151,10cyclictest13021-21latency_hist07:35:010
1182999141,9cyclictest0-21swapper/308:50:043
1182999141,9cyclictest0-21swapper/308:50:043
1182999141,9cyclictest0-21swapper/308:15:053
1180799141,9cyclictest13589-21munin-plugin-st07:50:001
1180799141,4cyclictest0-21swapper/109:01:271
1180799141,10cyclictest11944-21cron07:15:011
1179899146,5cyclictest12-21ksoftirqd/008:08:060
1179899141,5cyclictest12-21ksoftirqd/008:28:070
1182999138,3cyclictest0-21swapper/309:08:573
1182999138,3cyclictest0-21swapper/308:34:543
1180799131,9cyclictest11855-21latency_hist07:10:021
1182999127,3cyclictest36066irq/59-eth007:46:433
1182999127,3cyclictest33-21ksoftirqd/307:10:263
1182999127,3cyclictest0-21swapper/307:44:363
1182999127,3cyclictest0-21swapper/307:44:353
1182999122,7cyclictest0-21swapper/308:50:013
1182999122,6cyclictest0-21swapper/309:05:013
1182999111,7cyclictest0-21swapper/308:30:023
1182999111,7cyclictest0-21swapper/308:25:013
1182999111,7cyclictest0-21swapper/308:00:013
1182999111,7cyclictest0-21swapper/307:35:013
1182999102,3cyclictest0-21swapper/308:44:123
1182999101,7cyclictest0-21swapper/307:40:013
1182999101,6cyclictest0-21swapper/308:10:013
1182999101,6cyclictest0-21swapper/307:20:013
1182999101,6cyclictest0-21swapper/307:20:013
1182999101,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional