Skip to content

DRILL-4841: Use server event loop for web clients #565

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sudheeshkatkam wants to merge 3 commits into
base: master
Choose a base branch
from
Open

DRILL-4841: Use server event loop for web clients #565

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0889aa6
DRILL-4606: HYGIENE
Mar 9, 2016
8cc6bc9
DRILL-4606: CORE
Apr 15, 2016
b5ee0fb
DRILL-4841: Use server event loop for web clients
Aug 10, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 26 additions & 1 deletion common/src/main/java/org/apache/drill/common/AutoCloseables.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ public static void close(AutoCloseable... autoCloseables) throws Exception {

/**
* Closes all autoCloseables if not null and suppresses subsequent exceptions if more than one
* @param autoCloseables the closeables to close
* @param ac the closeables to close
*/
public static void close(Iterable<? extends AutoCloseable> ac) throws Exception {
Exception topLevelException = null;
Expand All @@ -87,4 +87,29 @@ public static void close(Iterable<? extends AutoCloseable> ac) throws Exception
throw topLevelException;
}
}

/**
* close() an {@see java.lang.AutoCloseable} without throwing a (checked)
* {@see java.lang.Exception}. This wraps the close() call with a
* try-catch that will rethrow an Exception wrapped with a
* {@see java.lang.RuntimeException}, providing a way to call close()
* without having to do the try-catch everywhere or propagate the Exception.
*
* @param autoCloseable the AutoCloseable to close; may be null
* @throws RuntimeException if an Exception occurs; the Exception is
* wrapped by the RuntimeException
*/
public static void closeNoChecked(final AutoCloseable autoCloseable) {
Copy link
Copy Markdown
Contributor

@paul-rogers paul-rogers Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

closeUnchecked ?

But, we are "checking", the "checked" has to do with the Exception type.

Maybe cleanClose( ) for this function. Then, add a "closeSilently" to catch and ignore close exceptions. (The closeSilently is handy in the case when, say, a file is full, a write failed, and the close will also fail because it still can't flush pending buffers.) There are other functions to the to silent close, but might be handy to have them in one place.

if (autoCloseable != null) {
try {
autoCloseable.close();
} catch(final Exception e) {
throw new RuntimeException("Exception while closing", e);
}
}
}

// prevents instantiation
private AutoCloseables() {
}
}
50 changes: 0 additions & 50 deletions common/src/main/java/org/apache/drill/common/DrillAutoCloseables.java
View file
Open in desktop

This file was deleted.

7 changes: 3 additions & 4 deletions exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@
import io.netty.channel.EventLoopGroup;

import java.io.Closeable;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
Expand All @@ -36,7 +35,7 @@
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

import org.apache.drill.common.DrillAutoCloseables;
import org.apache.drill.common.AutoCloseables;
import org.apache.drill.common.config.DrillConfig;
import org.apache.drill.common.exceptions.UserException;
import org.apache.drill.exec.ExecConstants;
Expand Down Expand Up @@ -302,8 +301,8 @@ public void close() {
if (this.client != null) {
this.client.close();
}
if (this.ownsAllocator && allocator != null) {
DrillAutoCloseables.closeNoChecked(allocator);
if (ownsAllocator && allocator != null) {
AutoCloseables.closeNoChecked(allocator);
}
if (ownsZkConnection) {
if (clusterCoordinator != null) {
Expand Down
4 changes: 2 additions & 2 deletions exec/java-exec/src/main/java/org/apache/drill/exec/client/PrintingResultsListener.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;

import org.apache.drill.common.DrillAutoCloseables;
import org.apache.drill.common.AutoCloseables;
import org.apache.drill.common.config.DrillConfig;
import org.apache.drill.common.exceptions.UserException;
import org.apache.drill.exec.client.QuerySubmitter.Format;
Expand Down Expand Up @@ -65,7 +65,7 @@ public void submissionFailed(UserException ex) {

@Override
public void queryCompleted(QueryState state) {
DrillAutoCloseables.closeNoChecked(allocator);
AutoCloseables.closeNoChecked(allocator);
System.out.println("Total rows returned : " + count.get() + ". Returned in " + w.elapsed(TimeUnit.MILLISECONDS)
+ "ms.");
}
Expand Down
4 changes: 2 additions & 2 deletions exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
import java.util.concurrent.TimeUnit;

import org.apache.calcite.rel.RelFieldCollation.Direction;
import org.apache.drill.common.DrillAutoCloseables;
import org.apache.drill.common.AutoCloseables;
import org.apache.drill.common.expression.ErrorCollector;
import org.apache.drill.common.expression.ErrorCollectorImpl;
import org.apache.drill.common.expression.LogicalExpression;
Expand Down Expand Up @@ -322,7 +322,7 @@ private void purge() throws SchemaChangeException {
builder.getSv4().clear();
selectionVector4.clear();
} finally {
DrillAutoCloseables.closeNoChecked(builder);
AutoCloseables.closeNoChecked(builder);
}
logger.debug("Took {} us to purge", watch.elapsed(TimeUnit.MICROSECONDS));
}
Expand Down
4 changes: 2 additions & 2 deletions exec/java-exec/src/main/java/org/apache/drill/exec/server/BootStrapContext.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

import org.apache.drill.common.DrillAutoCloseables;
import org.apache.drill.common.AutoCloseables;
import org.apache.drill.common.config.DrillConfig;
import org.apache.drill.common.scanner.persistence.ScanResult;
import org.apache.drill.exec.ExecConstants;
Expand Down Expand Up @@ -124,6 +124,6 @@ public void close() {
}
}

DrillAutoCloseables.closeNoChecked(allocator);
AutoCloseables.closeNoChecked(allocator);
}
}
11 changes: 8 additions & 3 deletions exec/java-exec/src/test/java/org/apache/drill/exec/DrillSeparatePlanningTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@
*/
package org.apache.drill.exec;

import static org.junit.Assert.*;
import io.netty.buffer.DrillBuf;

import java.util.Iterator;
Expand All @@ -28,7 +27,7 @@
import java.util.concurrent.ExecutionException;

import org.apache.drill.BaseTestQuery;
import org.apache.drill.common.DrillAutoCloseables;
import org.apache.drill.common.AutoCloseables;
import org.apache.drill.common.exceptions.UserException;
import org.apache.drill.common.util.TestTools;
import org.apache.drill.exec.client.DrillClient;
Expand Down Expand Up @@ -58,6 +57,12 @@
import com.google.common.collect.Lists;
import com.google.common.collect.Maps;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;

/**
* Class to test different planning use cases (separate form query execution)
*
Expand Down Expand Up @@ -304,7 +309,7 @@ public void queryIdArrived(QueryId queryId) {

@Override
public void submissionFailed(UserException ex) {
DrillAutoCloseables.closeNoChecked(allocator);
AutoCloseables.closeNoChecked(allocator);
this.ex = ex;
}

Expand Down
4 changes: 2 additions & 2 deletions exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestByteComparisonFunctions.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

import static org.junit.Assert.assertTrue;

import org.apache.drill.common.DrillAutoCloseables;
import org.apache.drill.common.AutoCloseables;
import org.apache.drill.common.config.DrillConfig;
import org.apache.drill.exec.ExecTest;
import org.apache.drill.exec.expr.fn.impl.ByteFunctionHelpers;
Expand Down Expand Up @@ -56,7 +56,7 @@ public static void teardown() {
helloLong.buffer.release();
goodbye.buffer.release();
goodbyeLong.buffer.release();
DrillAutoCloseables.closeNoChecked(allocator);
AutoCloseables.closeNoChecked(allocator);
}

@Test
Expand Down
4 changes: 2 additions & 2 deletions exec/java-exec/src/test/java/org/apache/drill/exec/server/TestDrillbitResilience.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@
import org.apache.drill.BaseTestQuery;
import org.apache.drill.QueryTestUtil;
import org.apache.drill.SingleRowListener;
import org.apache.drill.common.AutoCloseables;
import org.apache.drill.common.concurrent.ExtendedLatch;
import org.apache.drill.common.DrillAutoCloseables;
import org.apache.drill.common.config.DrillConfig;
import org.apache.drill.common.exceptions.UserException;
import org.apache.drill.common.types.TypeProtos.MinorType;
Expand Down Expand Up @@ -275,7 +275,7 @@ public void rowArrived(final QueryDataBatch queryResultBatch) {

@Override
public void cleanup() {
DrillAutoCloseables.closeNoChecked(bufferAllocator);
AutoCloseables.closeNoChecked(bufferAllocator);
}
};

Expand Down
4 changes: 2 additions & 2 deletions exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestRepeated.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@

import java.io.ByteArrayOutputStream;

import org.apache.drill.common.DrillAutoCloseables;
import org.apache.drill.common.AutoCloseables;
import org.apache.drill.common.config.DrillConfig;
import org.apache.drill.exec.expr.holders.BigIntHolder;
import org.apache.drill.exec.expr.holders.IntHolder;
Expand Down Expand Up @@ -52,7 +52,7 @@ public static void setupAllocator() {

@AfterClass
public static void destroyAllocator() {
DrillAutoCloseables.closeNoChecked(allocator);
AutoCloseables.closeNoChecked(allocator);
}
//
// @Test
Expand Down
4 changes: 2 additions & 2 deletions exec/memory/base/src/test/java/org/apache/drill/exec/memory/TestEndianess.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
import static org.junit.Assert.assertEquals;
import io.netty.buffer.ByteBuf;

import org.apache.drill.common.DrillAutoCloseables;
import org.apache.drill.common.AutoCloseables;
import org.apache.drill.common.config.DrillConfig;
import org.junit.Test;

Expand All @@ -37,7 +37,7 @@ public void testLittleEndian() {
assertEquals(b.getByte(2), 0);
assertEquals(b.getByte(3), 0);
b.release();
DrillAutoCloseables.closeNoChecked(a);
AutoCloseables.closeNoChecked(a);
}

}